sub return values are inaccurately documented
Xavier Noria
it was concluded the sentence in perlsub "The return value of a
subroutine is the value of the last expression evaluated by that sub"
is just not right.
Everyone around here knows counterexamples, but for the sake of the
discussion just take
sub foo { 1 for 1 }
The body of foo has two expressions, namely "1" and "1", and both are
evaluated at least once. So by definition "the last expression
evaluated" should be one of them. But foo() returns !1.
What is true is that all subroutines in Perl return _some_ value due
to the underlying implementation, but we need to be more precise
about which one, maybe leaving some cases unspecified (unless that
current not-quite-right statement in perlsub is accepted as good
enough).
Would that be a documentation or code patch?
-- fxn
Adriano Ferreira
> just take
> sub foo { 1 for 1 }
> The body of foo has two expressions, namely "1" and "1", and both are
> evaluated at least once. So by definition "the last expression
> evaluated" should be one of them. But foo() returns !1.
I think "sub return values are inaccurately documented" is hardly the
point. I guess that subroutines correctly return the value of the last
expression, but then what the control constructions like 'if',
'while', 'for' and so on return?
$ perl -MYAML -e "sub foo { 1 if (0) }; print Dump foo()"
--- 0
$ perl -MYAML -e "sub foo { 1 if (2) }; print Dump foo()"
--- 1
$ perl -MYAML -e "sub foo { 1 while (0) }; print Dump foo()"
--- 0
$ perl -MYAML -e 'sub foo { $a = 2; 1 while ($a--) }; print Dump foo()'
--- 0
$ perl -MYAML -e 'sub foo { 1 while (undef) }; print Dump foo()'
--- ~
$ perl -MYAML -e 'sub foo { 1 for 1 }; print Dump foo()'
--- ''
$ perl -MYAML -e 'sub foo { 1 for (1) }; print Dump foo()'
--- ''
$ perl -MYAML -e 'sub foo { 1 for (1, 2) }; print Dump foo()'
--- ''
suggests
* "if (cond) exp;" return cond if cond if false and exp if cond is true
* "while (cond) { exp }" return the false value of cond that makes
the loop end
* "for (list) { exp }" enjoys returning '' when the loop is over
Of course, there is more: if's with else's, while's with last's,
C-style for's, etc.
Maybe the return values of these constructions should be documented,
if they're not somewhere (which is probable - Rafael?). But then it is
not such a good idea if you don't fully understand what a statement
like this can return and don't use a explicit return (or at least an
expression whose value is obvious).
Michael Schwern commented about such issue in a recent p5p thread
http://www.xray.mpe.mpg.de/mailing-lists/perl5-porters/2005-09/msg00669.html
where missing explicit returns were causing trouble with hash vs subs
expressions.
We can paraphrase Larry's quote
If you want your program to be readable, consider supplying the argument.
with
If you want your program to be readable, consider using explicit returns.
Regards,
Adriano
Abigail
>
> Maybe the return values of these constructions should be documented,
Yes, but don't document it in perlsub. The subject of this thread is
a misnomer - what really is the issue is the return value of the constructs -
not of the sub.
$var = do {1 for 1};
has the same "problem", and it has nothing to do with return values of
a sub.
Abigail
Adriano Ferreira
> Yes, but don't document it in perlsub.
Agreed. They don't belong to perlsub. Maybe perlsyn?
Dave Mitchell
I don't think they should be documented anywhere, except to say that
the value of a control statement is unspecified and may change from
release to release.
--
"Strange women lying in ponds distributing swords is no basis for a system
of government. Supreme executive power derives from a mandate from the
masses, not from some farcical aquatic ceremony."
-- Dennis - Monty Python and the Holy Grail.
Xavier Noria
> On 10/20/05, Xavier Noria <f...@hashref.com> wrote:
>
>> just take
>> sub foo { 1 for 1 }
>> The body of foo has two expressions, namely "1" and "1", and both are
>> evaluated at least once. So by definition "the last expression
>> evaluated" should be one of them. But foo() returns !1.
>>
>
> I think "sub return values are inaccurately documented" is hardly the
> point. I guess that subroutines correctly return the value of the last
> expression
Didn't I provide a counterexample? Can you point out the error in my
reasoning?
In the example I sent there are only two expressions in the body of a
subroutine, neither of them being !1, which is the value returned by
a call. That is the argument that proves perlsub is not accurate
there, hence either perlsub or the relevant code need to be fixed. In
particular, I don't think the subject is a misnomer (assuming I am
right).
> We can paraphrase Larry's quote
> If you want your program to be readable, consider supplying
> the argument.
> with
> If you want your program to be readable, consider using
> explicit returns.
Absolutely, but we are talking about a formal issue in the language
here, not about best practices.
-- fxn
Adriano Ferreira
> >> just take
> >> sub foo { 1 for 1 }
> >> The body of foo has two expressions, namely "1" and "1", and both are
> >> evaluated at least once. So by definition "the last expression
> >> evaluated" should be one of them. But foo() returns !1.
> reasoning?
I believe your "sub foo" returns the value of the last
statement/expression which is "1 for 1". Seemingly the value of the
for statement is '' (not the value of the body or of the condition).
And that seems to be consistent.
If you say "for" is not an expression, but a control statement, it
remains the question: What is the last expression evaluated? Well,
that's up to how "for" is implemented. In this regard, Dave Mitchell's
suggestion about documenting the value of a control statement as
undefined is sensible. It prevents imposing an artificial constraint
to the implementation of the control construction, which would need to
work things out to return whatever it was imposed to return, instead
of just choosing the best/fast/shorter way to do what it is supposed
to (and that does not include return a value).
Adriano.
John Peacock
> In the example I sent there are only two expressions in the body of a
> subroutine, neither of them being !1, which is the value returned by a
> call.
You are mistaking the Perl code, as written, and the evaluation order of
the interpreted code itself. The last evaluated term may not be obvious
at first glance (which is what makes it such a b**ch to document). Your
example:
sub foo { 1 for 1 }
works like this (AFAICT) - for() loops over the inputs and exits _after_
the end of that list. And it turns out that the last eval() is of the
end of the list (an empty value).
How about I create a patch which issues a mandatory warning for all
sub{}'s defined without an explicit return()? And a matching patch
which silences the warnings in all core-included modules? And send
patches for each and every CPAN module that issues the warning?
:-J
John
p.s. :-J = "Tongue in cheek"
--
John Peacock
Director of Information Research and Technology
Rowman & Littlefield Publishing Group
4501 Forbes Boulevard
Suite H
Lanham, MD 20706
301-459-3366 x.5010
fax 301-429-5748
Xavier Noria
> After a couple of days discussing related issues in several places,
> it was concluded the sentence in perlsub "The return value of a
> subroutine is the value of the last expression evaluated by that
> sub" is just not right.
>
> Everyone around here knows counterexamples, but for the sake of the
> discussion just take
>
> sub foo { 1 for 1 }
>
> The body of foo has two expressions, namely "1" and "1", and both
> are evaluated at least once. So by definition "the last expression
> evaluated" should be one of them. But foo() returns !1.
Just to make my intention more clear, I don't mean we have to
document what control structures do behind the scenes and why that
value ends up being returned. I would leave that unspecified (maybe
explicitly), as Dave suggest. I would only document what subs return
in _regular usages_ like explicit returns and regular uses of
expressions as last things to evaluate. The current "last expression
evaluated" is just wrong.
-- fxn
Xavier Noria
> On 10/20/05, Xavier Noria <f...@hashref.com> wrote:
>
>>>> just take
>>>> sub foo { 1 for 1 }
>>>> The body of foo has two expressions, namely "1" and "1", and
>>>> both are
>>>> evaluated at least once. So by definition "the last expression
>>>> evaluated" should be one of them. But foo() returns !1.
>>>>
>> Didn't I provide a counterexample? Can you point out the error in my
>> reasoning?
>>
>
> I believe your "sub foo" returns the value of the last
> statement/expression which is "1 for 1". Seemingly the value of the
> for statement is '' (not the value of the body or of the condition).
> And that seems to be consistent.
>
> If you say "for" is not an expression, but a control statement,
Of course, that is the underlying argument. We've got two expressions
and perlsub talks only of expressions. That's all.
> it
> remains the question: What is the last expression evaluated? Well,
> that's up to how "for" is implemented.
Whichever is last evaluated, my example proves none of the
expressions is returned. The last evaluated expression would be
implementation-dependent in that case in a for loop, but we are not
getting even an expression!
What we are getting is something else as a consequence of how the
interpreter is implemented, but nothing close to what is documented
in perlsub.
-- fxn
Xavier Noria
> Xavier Noria wrote:
>
>> In the example I sent there are only two expressions in the body
>> of a subroutine, neither of them being !1, which is the value
>> returned by a call.
>>
>
> You are mistaking the Perl code, as written, and the evaluation
> order of the interpreted code itself.
If that is the point I would be wrong and my confusion would come
from a misconcetion about the formal jargon.
For me an expression is something formal, something to be parsed. The
expressions in
sub foo { foreach (@_) { 1 } }
are only "@_" and "1", there are only two. The whole stuff the
interpreter evaluates to execute that body is for me a complete
different thing. In particular a hidden, internal boolean test has
nothing to do with the expressions in the code, which is what perlsub
mentions, expressions.
That is what the "EXPR" in "eval EXPR" means, and that's why I can't
put a foreach there.
-- fxn
John Peacock
> If that is the point I would be wrong and my confusion would come from
> a misconcetion about the formal jargon.
>
> For me an expression is something formal, something to be parsed.
Except that Perl is an _interpreted_ language. Once the program has
been parsed (using the formal grammer), the interpreter actually
executes the op tree (which may have undergone automatic optimization
and reordering).
The formal definition of sub {} allows you to employ an explicit return
(in which case the Perl code has to specify a return value or values),
or an implicit return. In the implicit case, the interpreter just
returns whatever it was that the last op returned. That last op may or
may not be related to the original Perl code as written. In the case of
for(), the last op is the loop itself, not any values in the loop, hence
it always returns '' (as someone else demonstrated).
The implicit return is an optimization which may not do what you expect
unless the last term to be evaluated is "trivial" like an assignment.
By necessity, it depends on how the interpreter processes the op tree.
That is why the use of the implicit return is not recommended (it is
also going to do the wrong thing in the case where the return value was
supposed to be a list).
John
Dave Mitchell
> sub foo { 1 for 1 }
>
> works like this (AFAICT) - for() loops over the inputs and exits _after_
> the end of that list. And it turns out that the last eval() is of the
> end of the list (an empty value).
Strickly speaking the value of the statment is PL_sv_no, which is the
value which pp_iter pushes on the stack to indicate 'no more iterations';
which is then also left on the stack by the pp_and op which tests the
condition to decide whether to jump to the end of the loop. This value
is normally then popped by the next pp_nextstate to be executed, except
that in this case there is no pp_nextstate op.
--
"There's something wrong with our bloody ships today, Chatfield."
-- Admiral Beatty at the Battle of Jutland, 31st May 1916.
John Peacock
> The only problem comes when its not an expression, and currently the
> documentation doesn't say what happens in this case.
That's the case I was referring to; sorry I should have been more explicit.
Dave Mitchell
> The implicit return is an optimization which may not do what you expect
> unless the last term to be evaluated is "trivial" like an assignment.
> By necessity, it depends on how the interpreter processes the op tree.
> That is why the use of the implicit return is not recommended (it is
> also going to do the wrong thing in the case where the return value was
> supposed to be a list).
Huh?
An implicit return works fine, even as a list, as long as the last
statement to execute is an expression, eg
$ perl -we 'sub f { (1,2,3) } @a = f(); print "@a\n"'
1 2 3
$
The only problem comes when its not an expression, and currently the
documentation doesn't say what happens in this case.
--
Spock (or Data) is fired from his high-ranking position for not being able
to understand the most basic nuances of about one in three sentences that
anyone says to him.
-- Things That Never Happen in "Star Trek" #19
John Peacock
> Strickly speaking the value of the statment is PL_sv_no, which is the
Yeah, what he said!
Having been through hours of intensive therapy to forget what little I
knew of the pp_* code, I bow down to Dave's superior knowledge. ;-)
John
Xavier Noria
> Xavier Noria wrote:
>
>> If that is the point I would be wrong and my confusion would come
>> from a misconcetion about the formal jargon.
>> For me an expression is something formal, something to be parsed.
>>
>
> Except that Perl is an _interpreted_ language. Once the program
> has been parsed (using the formal grammer), the interpreter
> actually executes the op tree (which may have undergone automatic
> optimization and reordering).
Sure, but then perlsub would have to say "the last thing the
interpreter evaluates internally". That would be obscure but more
accurate (I say that for the sake of the discussion, not that I am
proposing it). The issue is that it reads "the last evaluated
expression", and you can count expressions in a listing.
If you count more than two expressions in the listing
foreach (@foo) {
1;
}
then we are using a different definition of expression. If you count
two perlsub needs to be fixed somehow.
-- fxn
Yitzchak Scott-Thoennes
For my take on this, see http://perlmonks.org/?node=498843
I would like to see some improvement in what statements return,
including a test suite, and once it is more consistent document it.
IMO a for statement should return nothing (becoming undef in scalar
context).
Note that last I checked there are also problems with optimized-away
unless(something-false) not returning that particular something-false.
Demerphq
FWIW I think I agree with Xavier on this. The docs dont say it should
return the result of the last opcode evaluated they say "The return
value of a subroutine is the value of the last expression evaluated."
Which is why sub foo { if (shift) { 1 } else { 0 } } is expected to
return 1 if the argument is true and 0 if it is false or omitted.
The level of abstraction of foreach (@list) {} is that one tends not
to consider it an analog of a for(;;) or while(), so unless it is
formally defined as such id consider do { 1 for 1 } to return 1 and do
{ for (1) {} } to return undef/empty list and do { for (1) { 0 } } to
return 0.
Also, while i was playing around with this I noticed that
d:\>perl -we " @ok=(0,1); print $ok[ do { 1 for 1 } ]"
prints nothing!, and
d:\>perl -we " @ok=(0,1); print $ok[ 0 + do { 1 for 1 } ]"
segfaults with 5.8.6! So Id say there is a bug here somewhere. Other oddness:
d:\>perl -we " @ok=(0,1); print $ok[ !do { 1 for 1 } ]"
d:\>perl -we " @ok=(0,1); print $ok[ !!do { 1 for 1 } ]"
d:\>perl -v
This is perl, v5.8.6 built for MSWin32-x86-multi-thread
(with 3 registered patches, see perl -V for more detail)
Copyright 1987-2004, Larry Wall
Binary build 811 provided by ActiveState Corp. http://www.ActiveState.com
ActiveState is a division of Sophos.
Built Dec 13 2004 09:52:01
--
perl -Mre=debug -e "/just|another|perl|hacker/"
Xavier Noria
> FWIW I think I agree with Xavier on this. The docs dont say it should
> return the result of the last opcode evaluated they say "The return
> value of a subroutine is the value of the last expression evaluated."
> Which is why sub foo { if (shift) { 1 } else { 0 } } is expected to
> return 1 if the argument is true and 0 if it is false or omitted.
This is the crucial point in my view, we don't give control
structures the range of expressions AND at the same time we are
trying to DWIM in some places expect and USE some value from them. As
if that was robust.
Balancing this in a coherent way looks difficult to me, in Ruby for
instance this is easier because an if/else IS an expression, you can
assign an if/else to a variable.
Either we document somewhere the value returned by all control
structures and document WHERE that value can be used, or else we
forget about it and say non-empty subroutines return an unspecified
value if the last statement is not an expression (which is what my
patch does). Nothing related to values returned by control structures
at all, we don't even mention such a thing, the focus is in the
subroutine return value.
-- fxn
Demerphq
> Either we document somewhere the value returned by all control
> structures and document WHERE that value can be used, or else we
> forget about it and say non-empty subroutines return an unspecified
> value if the last statement is not an expression (which is what my
> patch does). Nothing related to values returned by control structures
> at all, we don't even mention such a thing, the focus is in the
> subroutine return value.
The problem i have is that bringing up statements in this conversation
is more or less a red herring. Statements play no role in determining
what is returned from a subroutine. If they do then its a bug. Its not
an optimisation, its not undefined behaviour, its a BUG.
The argument that the internal flag that is used by perl to indicate
that the list traversal is finished is an EXPR is just wrong. Thats an
internal flag that has no business being visible from the POV of perl
code.
This talk that having a statement as the last code in a subroutine
without an explicit return renders the return of the subroutine
undefined is just wrong and would render massive amounts of code out
there to be undefined, a situation that I really hope everybody
reading this agrees is totally unacceptable.
The docs are simple: they say the return value is that of the last
expression evaluated. They dont speak of statements, and for good
reason. The docs dont say "if the last statement executed in a
subroutine is also an EXPR then the returned value is that of the
EXPR." They leave out statements deliberately because they shouldnt
matter at all.
The solutuion to this is to fix the behaviour of foreach so that the
internal flag is not visible from perl. If this is too difficult then
we should document foreach as a buggy special case which has undefined
results. Either way do { 1 for 1} is returning something strange and
can be used to cause a segfault, so the behaviour is definately BROKEN
currently.
E:\uedit32>perl -MDevel::Peek -we "Dump do{1 for 1}"
SV = PVNV(0x184a36c) at 0x2258d8
REFCNT = 1
FLAGS = (TEMP,IOK,NOK,POK,pIOK,pNOK,pPOK)
IV = 0
NV = 0
PV = 0x22d3d4 ""\0
CUR = 0
LEN = 1
E:\uedit32>perl -MDevel::Peek -we "Dump 0+do{1 for 1}"
Use of uninitialized value in addition (+) at -e line 1.
Usage: Devel::Peek::Dump(sv, lim=4) at -e line 1.
E:\uedit32>perl -MDevel::Peek -we "Dump(0+do{1 for 1})"
Use of uninitialized value in addition (+) at -e line 1.
Usage: Devel::Peek::Dump(sv, lim=4) at -e line 1.
E:\uedit32>perl -MDevel::Peek -we "Dump(1+do{1 for 1})"
Use of uninitialized value in addition (+) at -e line 1.
Usage: Devel::Peek::Dump(sv, lim=4) at -e line 1.
Yitzchak Scott-Thoennes
Fix how? If you mean changing the last expression *inside* the for
block to propagate the external context instead of having void context,
that is also a backward-compatibility concern, this time in actual
functionality, not just in the doc.
The alternative (and a much better one IMO) is to recognize that
statements *do* matter. We should decide what a for (LIST) statement
*should* return and make it so (assuming other people share my opinion
that sv_no is not the correct thing).
Then we should develop tests for the expected return of all types of
statements. And then we should document it. But in the meantime,
adding a warning in the doc that it is undefined and subject to change
sounds ok to me. Obviously that won't mean that simple cases such as
if/else are subject to change; if that could be mentioned without
getting excessively verbose, it should be.
Michael G Schwern
> > Agreed. They don't belong to perlsub. Maybe perlsyn?
>
> I don't think they should be documented anywhere, except to say that
> the value of a control statement is unspecified and may change from
> release to release.
Seconded.
--
Michael G Schwern sch...@pobox.com http://www.pobox.com/~schwern
...they shared one last kiss that left a bitter yet sweet taste in her
mouth--kind of like throwing up after eating a junior mint.
-- Dishonorable Mention, 2005 Bulwer-Lytton Fiction Contest
by Tami Farmer