Autovivi
Deven T. Corzine
> At 1:30 PM +1000 8/6/02, Damian Conway wrote:
> >Luke Palmer asked:
> >
> >>Does:
> >>
> >> print %foo{bar}{baz};
> >>
> >>still create %foo{bar} as a hashref if it doesn't exist?
> >
> >It is my very strong hope that it will not.
>
> Unless Larry declares otherwise, it won't.
I bought up this question on 12/22/00, and it didn't get much response,
except for Graham Barr, who pointed out that there might be some trouble
with constructs like "func($x{1}{2}{3})" that may or may not need to
autovivify the argument if the function assigns to $_[0]. (This seems a
bit like the difficulty of implementing "hypothetical" values in regexes;
perhaps a similar solution could apply to both?)
Subsequent to that mini-discussion, I had occasion to talk to Larry in
person, and asked him about the autovivification behavior -- his response
was that he considered Perl 5's behavior of autovivifying read-only values
to be a misfeature, and that he hopes Perl 6 won't do so.
Now, since Larry, Dan and Damian all seem to be in agreement on this issue,
I sincerely hope Perl 6 will indeed break from Perl 5's misbehavior here...
However, will the "func($x{1}{2}{3})" case cause an implementation problem?
Deven
Larry Wall
: However, will the "func($x{1}{2}{3})" case cause an implementation problem?
This is why the new function type signatures will assume that
parameters are constant. If you want a modifiable parameter, you
have to say "is rw". Then it's considered an lvalue. In Perl 5,
all function args had to be considered potential lvalues.
In essence, all Perl 5 functions have a signature of (*@_ is rw).
Perhaps the translator can turn some of those into (*@_). What'd
really be cool is if it could pick up an initial
my ($a, $b, $c) = @_;
and turn it into an appropriate signature. Of course, there are issues
here if the code modifies those variables, since the issue of whether
a variable is rw is really distinct from whether it represents a pass
by value or reference. Slapping a "constant" on it is a bald-faced
attempt to get the speed of pass-by-reference with the guarantees of
pass-by-value. Perhaps there should be a way to declare a parameter
to be pass-by-value, producing a modifiable variable that does not
affect the caller's value. But I'm not sure saving one assignment
in the body is worth the extra mental baggage.
Larry
mosul...@crtinc.com
> Of course, there are issues here if the code modifies those
> variables, since the issue of whether a variable is rw is
> really distinct from whether it represents a pass by value
> or reference. Slapping a "constant" on it is a bald-faced
> attempt to get the speed of pass-by-reference with the
> guarantees of pass-by-value. Perhaps there should be a way
> to declare a parameter to be pass-by-value, producing a
> modifiable variable that does not affect the caller's value.
> But I'm not sure saving one assignment in the body is worth
> the extra mental baggage.
I'm not sure I read that right, but it sounds like you're implying that
pass-by-values will be read-only. Is that the plan?
-Miko
--------------------------------------------------------------------
mail2web - Check your email from the web at
http://mail2web.com/ .
Deven T. Corzine
> On Tue, 13 Aug 2002, Deven T. Corzine wrote:
> : However, will the "func($x{1}{2}{3})" case cause an implementation problem?
>
> This is why the new function type signatures will assume that
> parameters are constant. If you want a modifiable parameter, you
> have to say "is rw". Then it's considered an lvalue. In Perl 5,
> all function args had to be considered potential lvalues.
That will certainly help. What about the "is rw" case, though? Will we be
able to avoid autovivifying the parameter if it's never actually assigned,
or do we assume this edge case isn't worth the hassle, and assume it will
follow Perl 5's behavior for such variables?
> In essence, all Perl 5 functions have a signature of (*@_ is rw).
> Perhaps the translator can turn some of those into (*@_). What'd
> really be cool is if it could pick up an initial
>
> my ($a, $b, $c) = @_;
>
> and turn it into an appropriate signature.
That would be pretty cool. It would also be nice to handle the common
alternative form of using "shift" as well...
> Of course, there are issues here if the code modifies those variables,
> since the issue of whether a variable is rw is really distinct from
> whether it represents a pass by value or reference. Slapping a
> "constant" on it is a bald-faced attempt to get the speed of
> pass-by-reference with the guarantees of pass-by-value.
The only accurate way to know if the code modifies the variables is to do
some sort of dataflow analysis, and it can't be 100% accurate even then.
(Suppose a "shift" may or may not happen, depending on the parameters, then
$_[2] is modified?) Of course, it should often be possible (at least in
principle) to determine that it's impossible for a particular parameter to
be modified, and play it safe by assuming "is rw" for undeterminable cases.
> Perhaps there should be a way to declare a parameter to be pass-by-value,
> producing a modifiable variable that does not affect the caller's value.
> But I'm not sure saving one assignment in the body is worth the extra
> mental baggage.
I'd argue that it's worthwhile -- not because of the cost of that single
assignment, but because you'd now have two variables to keep track of, the
original parameter and the modifiable copy. This seems like greater mental
bagger (to me) than remembering another modifier.
How about "is copied" as a pass-by-value modifier? (I was inclined to say
"is a copy" but I didn't think a 3-word phrase would be desirable here...)
Deven
Larry Wall
: From: Larry Wall la...@wall.org
: > Of course, there are issues here if the code modifies those
: > variables, since the issue of whether a variable is rw is
: > really distinct from whether it represents a pass by value
: > or reference. Slapping a "constant" on it is a bald-faced
: > attempt to get the speed of pass-by-reference with the
: > guarantees of pass-by-value. Perhaps there should be a way
: > to declare a parameter to be pass-by-value, producing a
: > modifiable variable that does not affect the caller's value.
: > But I'm not sure saving one assignment in the body is worth
: > the extra mental baggage.
:
: I'm not sure I read that right, but it sounds like you're implying that
: pass-by-values will be read-only. Is that the plan?
The default is pass-by-reference, but non-modifiable. If there's a
pass-by-value, it'll have to be specially requested somehow.
This is a minimal difference from Perl 5, in which everything was
pass-by-reference, but modifiable. To get pass-by-value, you just
do an assignment. :-)
Larry
Mike Lambert
> Perhaps the translator can turn some of those into (*@_). What'd
> really be cool is if it could pick up an initial
>
> my ($a, $b, $c) = @_;
Excuse my ignorance here, but I thought the plan for backwards
compatibility with perl5 was to embed the Perl5 engine with Parrot, and
create thunking/compatibility interfaces between the two.
If we try to create a converter to convert perl5 to perl6 code, by say
using a B:: backend or such, it forces us to develop both perl5
*and* perl6 in Parrot, each with their own PMCs and semantics. Not to
mention the amount of intelligence and introspective abilities you'll need
to program into that convertor to actually be able to do the things that
are needed. A simple translator will write code which assumes that 'eval'
can be called anywhere, etc. Which might very well lead to performance
that's worse than perl5 (especially if we force DOD runs on every block
exit to ensure the same liveness guarantees of refcounting.)
Does anyone actually know what the plan for perl5 to perl6 conversion is,
or are we just assuming all possible avenues now, to be discarded as we
approach the implementation phase and realize the scope of the project. :)
Thanks,
Mike Lambert
Nicholas Clark
> The only accurate way to know if the code modifies the variables is to do
> some sort of dataflow analysis, and it can't be 100% accurate even then.
> (Suppose a "shift" may or may not happen, depending on the parameters, then
> $_[2] is modified?) Of course, it should often be possible (at least in
> principle) to determine that it's impossible for a particular parameter to
> be modified, and play it safe by assuming "is rw" for undeterminable cases.
Well, perl5 does already manage to avoid auto-vivifying hash keys when they
are used as subroutine arguments. It uses magic, rather than dataflow
analysis:
$ cat autovivi.pl
#!/usr/local/bin/perl -w
use strict;
use Devel::Peek;
sub rval {
my $a = $_[0];
}
sub lval {
$_[0] = undef;
}
sub dumpit {
Dump ($_[0]);
}
my %a = (camel => 2);
rval ($a{llama});
print "rval: @{[keys %a]}\n";
lval ($a{alpaca});
print "lval: @{[keys %a]}\n\n";
dumpit ($a{dromedary});
__END__
$ ./autovivi.pl
rval: camel
lval: camel alpaca
SV = PVLV(0x1134d8) at 0xf4368
REFCNT = 1
FLAGS = (GMG,SMG)
IV = 0
NV = 0
PV = 0
MAGIC = 0x113420
MG_VIRTUAL = &PL_vtbl_defelem
MG_TYPE = PERL_MAGIC_defelem(y)
MG_FLAGS = 0x02
REFCOUNTED
MG_OBJ = 0xf43a4
SV = PVIV(0xf4990) at 0xf43a4
REFCNT = 1
FLAGS = (POK,pPOK)
IV = 0
PV = 0x1138a0 "dromedary"\0
CUR = 9
LEN = 10
TYPE = y
TARGOFF = 0
TARGLEN = 1
TARG = 0x1329ec
SV = PVHV(0x114820) at 0x1329ec
REFCNT = 2
FLAGS = (PADBUSY,PADMY,SHAREKEYS)
IV = 2
NV = 0
ARRAY = 0x1133f8 (0:6, 1:2)
hash quality = 125.0%
KEYS = 2
FILL = 2
MAX = 7
RITER = -1
EITER = 0x0
Elt "camel" HASH = 0x787f96b8
SV = IV(0x112ee4) at 0xf4284
REFCNT = 1
FLAGS = (IOK,pIOK)
IV = 2
Elt "alpaca" HASH = 0x5af8b041
SV = PVNV(0x12a6d8) at 0x111b40
REFCNT = 1
FLAGS = ()
IV = 0
NV = 0
PV = 0
As you can see, the ingredients for "magic" are quite cumbersome, but do get
the job done.
Nicholas Clark
--
Even better than the real thing: http://nms-cgi.sourceforge.net/
Deven T. Corzine
> On Tue, Aug 13, 2002 at 03:06:40PM -0400, Deven T. Corzine wrote:
>
> > The only accurate way to know if the code modifies the variables is to do
> > some sort of dataflow analysis, and it can't be 100% accurate even then.
> > (Suppose a "shift" may or may not happen, depending on the parameters, then
> > $_[2] is modified?) Of course, it should often be possible (at least in
> > principle) to determine that it's impossible for a particular parameter to
> > be modified, and play it safe by assuming "is rw" for undeterminable cases.
>
> Well, perl5 does already manage to avoid auto-vivifying hash keys when they
> are used as subroutine arguments. It uses magic, rather than dataflow
> analysis:
Yes, "magic" obviously can work, but it's a greater run-time penalty -- if
you do some sort of dataflow analysis, you pay the penalty at compile time
instead. I'm not sure which is better, ultimately. It probably varies.
Anyway, the Perl 5 magic you speak of presumably only works for a single
level of hash referencing -- we were specifically discussing a multilevel
reference, which immediately creates missing substructures in Perl 5, even
used in a read-only rvalue context in a simple Perl 5 expression...
Deven
Uri Guttman
NC> Well, perl5 does already manage to avoid auto-vivifying hash keys
NC> when they are used as subroutine arguments. It uses magic, rather
NC> than dataflow analysis:
NC> sub rval {
NC> my $a = $_[0];
NC> }
NC> sub lval {
NC> $_[0] = undef;
NC> }
NC> sub dumpit {
NC> Dump ($_[0]);
NC> }
NC> my %a = (camel => 2);
NC> rval ($a{llama});
that is only one level deep and perl never autovivified that when
reading. try this with $a{llama}{alpaca}. that will autoviv the 'llama'
level even when reading. i don't think the sub has anything to do with
it. the expression is evaled before the pass by ref is done and the ref
is to the lowest element. so the higher elements must be autovived to
make that work. i know why larry and others want this to change but it
will be tricky as you can't know how this ref will be used later
on. what if you passed \$a{llama}{alpaca}? even as a read only param,
you could deref later through the ref in another sub that gets passed it
from this sub. but how do you pass a ref to hash levels that don't
exist? lazy eval may be one way but then you have to pass a thunk (what
algol called pass by name/code thingys) and eval that at the time of the
deref.
this is a minor can of worms IMO. we can define the language behavior to
not autoviv on reading but how to detect that at runtime is an issue. i
think larry didn't do this in perl5 more because of the implementation
was very hard rather than for the poor choice of semantics.
uri
--
Uri Guttman ------ u...@stemsystems.com -------- http://www.stemsystems.com
----- Stem and Perl Development, Systems Architecture, Design and Coding ----
Search or Offer Perl Jobs ---------------------------- http://jobs.perl.org
Leopold Toetsch
[ CCs stripped ]
> ... what if you passed \$a{llama}{alpaca}? even as a read only param,
> you could deref later through the ref in another sub that gets passed it
> from this sub.
If I understand Dan's proposal (<a05111b55b977c7a65606@[63.120.19.221]>)
for a change in the KEYed methods correctly, this would work, when
additionaly the KEY get's passed on with the PMC.
my %a;
func (%a{llama}{alpaca});
new P0, .PerlHash
new P1, .KEY
set P1[0], "llama"
set P1[1], "alpaca" # KEY chain P1{llama; alpaca}
setk? P0, P1 # assoc KEY chain P1 to P0
save P0
bsr func
func ($h is rw) {
print $h;
$h = 'wooly';
}
func:
pushp
restore P0
print P0 # get_p_keyed gets implicitly it's own KEY => undef
set P0, 'wooly' # set_p_keyed autovivs KEY (chain)
popp
ret
> ... but how to detect that at runtime is an issue.
If the PMC carrys it's KEY around, this should work.
> uri
leo
mosul...@crtinc.com
> Perhaps there should be a way
> to declare a parameter to be pass-by-value, producing a
> modifiable variable that does not affect the caller's value.
> But I'm not sure saving one assignment in the body is worth
> the extra mental baggage.
.... and later he said ...
> The default is pass-by-reference, but non-modifiable. If
> there's a pass-by-value, it'll have to be specially requested
> somehow.
>
> This is a minimal difference from Perl 5, in which everything
> was pass-by-reference, but modifiable. To get pass-by-value,
> you just do an assignment. :-)
Thesis: there should be an option for pass-by-value-modifiable without
requiring a new assignment or new variable.
Reason 1: Pass-by-value-non-modifiable *is* a significant change from
Perl5. I know this doesn't quite mesh with the technical facts, but from
the how-people-actually-use-it perspective, there's a big change. Yes,
Perl5 is pass-by-reference-modifiable, but in practice most people don't
use it that way. Except for short 1-3 line functions, most subs start by
creating named variables and copying them from @_, like one of these lines:
my $self = shift;
my ($tag) = @_;
That is, they actually use it like pass-by-value-modifiable. Now, Perl6
will give us a slicker way to get well-named variables as subroutine
arguments:
sub mysub ($name, $email) { ... }
Ah, but that's where a bait-and-switch problem comes in. It feels like the
param list is taking the place of "my ($tag) = @_;", but the new-improved
named arguments don't act the same way. Suddenly they're not modifiable.
That's different.
Reason 2: Enforced non-modifiable arguments are a pain. PL/SQL does not
allow you to modify pass-by-value arguments to functions, and most PLSQL
programmers I've talked to hate that. It requires you to create a named
argument, then another variable with an almost-same-name that can be
modified. My PLSQL is cluttered with variables named v_email and p_email,
instead of just email. I would hate to see Perl go down that road.
Resolution: Use whatever default seems good, but provide the freedom to get
pass-by-value-modifiable, perhaps something like this:
sub mysub ($name is m, $email is m) { ... }
Brent Dax
# Resolution: Use whatever default seems good, but provide the
# freedom to get pass-by-value-modifiable, perhaps something like this:
#
# sub mysub ($name is m, $email is m) { ... }
Of course! This *is* Perl after all--did you ever doubt that we would
give you all the flexibility you wanted? ;^)
--Brent Dax <bren...@cpan.org>
@roles=map {"Parrot $_"} qw(embedding regexen Configure)
He who fights and runs away wasted valuable running time with the
fighting.
Luke Palmer
> The default is pass-by-reference, but non-modifiable. If
> there's a pass-by-value, it'll have to be specially requested
> somehow.
>
> This is a minimal difference from Perl 5, in which everything
> was pass-by-reference, but modifiable. To get pass-by-value,
> you just do an assignment. :-)
Why? We could make arglists exactly equivilent to the way they're done in
Perl 5, which is a good way.
sub foo($a, $b, *@c) {...}
Would be exactly equivilent to Perl 5's
sub foo { my ($a, $b, @c) = @_; ... }
Since variables are copy-on-write, you get the speed of pass-by-reference
with the mutability of pass-by-value, which is what everyone wants. If you
have this, why would you want to do enforced const reference? That's not
rhetorical; I'm actually curious.
Luke
David Whipp
> Since variables are copy-on-write, you get the speed of
> pass-by-reference with the mutability of pass-by-value,
> which is what everyone wants. If you have this, why would
> you want to do enforced const reference? That's not
> rhetorical; I'm actually curious.
One reason I can think of would be if you had multiple
threads, and you wanted to poll a pass-by-reference
shared value inside the sub:
sub wait_for_done ( $done is shared )
{
return if $done;
sleep 1;
redo;
}
Dave.
Deven T. Corzine
> Why? We could make arglists exactly equivilent to the way they're done in
> Perl 5, which is a good way.
>
> sub foo($a, $b, *@c) {...}
>
> Would be exactly equivilent to Perl 5's
>
> sub foo { my ($a, $b, @c) = @_; ... }
I've got another idea. How about using a copy-restore technique?
First, assume all arguments are pass-by-reference, whether constant or not.
Second, allow "is ro" as a modifier to force constant semantics -- in this
case, don't make a copy, just use the reference and refuse to modify it.
Third, allow "is rw" as a modifier to enable copy-restore, and to make the
reference modifiable.
Fourth, copy all named parameters into local variables from the passed
reference (except for "is ro" parameters, which should be aliased) -- and
then keep @_ with the actual references, as in Perl 5. (Unlike Perl 5, the
"is ro" parameters would not be modifiable even via @_.)
Finally, for "is rw" parameters, copy the modified local variable back to
the referenced parameter (the original one, even if @_ was shifted!), but
ONLY for a normal subroutine exit.
This would allow pass-by-value semantics by default for convenience, easily
forced read-only parameters, pass-by-reference semantics when desired, but
with a twist -- if the function aborts (throws an exception), no parameters
have been changed, which offers hassle-free transaction-like atomicity.
If a function really wants to modify its parameter, despite throwing an
exception, it could do so via @_ directly. Or, we could also allow another
modifier "is ref" to use simple (modifiable) pass-by-reference semantics,
but default to copy-restore for "is rw"...
This seems like it could be the best of all worlds -- what do people think?
Deven
Larry Wall
: I've got another idea. How about using a copy-restore technique?
I suspect that would make Perl 6's sub calls even slower than Perl 5's.
Larry
Deven T. Corzine
Yes and no.
For the normal case (pass-by-value semantics), it should be about the same
speed as Perl 5, since it would be doing the same thing: making a copy of a
value that was passed by reference.
For the "is ro" or "is ref" case, it should be FASTER, since it could avoid
the copy by aliasing the reference.
Only the "is rw" case would be slower, since it would involve two copies
rather than one. However, it would offer a benefit of "rolling back" any
changes to the parameters in case of an abnormal exit from the function.
This seems like a valuable feature that could often be useful -- and anyone
who wants to avoid the speed penalty could stick with "is ref", assuming
that option is also allowed...
Deven
Luke Palmer
You're talking about exception safety. First of all, to have a
language feature automatically enforce exception safety is too
inefficient. The fastest way to do it is problem-specific. Stroustrup
wrote big papers discussing this.
The first issue: Methods. Even though your routines are safe to the
parameters, they still wouldn't be safe to attributes of the object, which
is often the kind of safety that's important anyway.
The second issue: Awful, awful speed. Take for example this sub:
sub randinc(@ar is rw) {
@ar[ int rand @ar ]++;
}
Where @ar has 5000 elements. You don't really want to copy that whole
array twice just to increment one element (There might be some internals
stuff going on that would make this normal). Note that this function is
already exception safe even without your copy-restore semantics.
I think I see where Larry's going with the read-only reference. Perl 5
sub calls are notoriously slow, so people are sometimes afraid to use them
a lot. So it's making them faster. That and there's well defined, easy
semantics; i.e. every time a copy is being made, it's explicit. I can see
the advantages of this.
But back to exception safety. Perl needs some way to handle exception
safety if it's going to be as powerful as we say. As far as large data
structures: will this run in constant time?
(@a, @b) = (@b, @a)
If so (and the same holds for hashes), then there's probably no need to
build in anything else besides CATCH blocks, etc.
That was a digression and a half. Sorry. :)
Luke
Peter Haworth
> We could make arglists exactly equivilent to the way they're done in Perl 5,
> which is a good way.
>
> sub foo($a, $b, *@c) {...}
>
> Would be exactly equivilent to Perl 5's
>
> sub foo { my ($a, $b, @c) = @_; ... }
>
> Since variables are copy-on-write, you get the speed of pass-by-reference
> with the mutability of pass-by-value, which is what everyone wants.
No you don't. Since the arguments have to be copied into the local
variables, you get the speed of pass-by-value along with its mutability.
That doesn't sound like what everyone wants to me.
> If you have this, why would you want to do enforced const reference?
Because it's the safest and fastest option. Of course this isn't what
everyone wants, either. However, by making the programmer explictly ask for
the other options (of which there are sevaral), we only give them exactly
what they want. Perl 5 gives you the most flexible way by default (pass by
ref, modifiable), and makes one other option (pass by val, modifiable) easy,
but has occassionally surprising results, such as autovivification.
--
Peter Haworth p...@edison.ioppublishing.com
E is for emacs, which rebinds your keys, and
F is for fsck, which rebuilds your trees.
G is for grep, a clever detective, while
H is for halt, which may seem defective.
Dan Sugalski
> > In essence, all Perl 5 functions have a signature of (*@_ is rw).
>> Perhaps the translator can turn some of those into (*@_). What'd
>> really be cool is if it could pick up an initial
>>
>> my ($a, $b, $c) = @_;
>
>Excuse my ignorance here, but I thought the plan for backwards
>compatibility with perl5 was to embed the Perl5 engine with Parrot, and
>create thunking/compatibility interfaces between the two.
Nope, not ultimately. B:: backend first, then a perl 5 parser with
requisite PMC classes.
--
Dan
--------------------------------------"it's like this"-------------------
Dan Sugalski even samurai
d...@sidhe.org have teddy bears and even
teddy bears get drunk