Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

New feature proposal : <<>> to disable magic open of ARGV

210 views
Skip to first unread message

Rafael Garcia-Suarez

unread,
Jul 24, 2014, 11:51:40 AM7/24/14
to perl5-...@perl.org
I believe that this was discussed here some years ago, and that this
was suggested by TomC. Anyway, I pushed a patch on rgs/nomagicopen.
The patch is incomplete as it lacks docs and tests. I welcome feedback
on the intent of the feature.

commit 7f5bb3418c4fc2d6e2523bf918a7d9e33f9a7acc
Author: Rafael Garcia-Suarez <r...@consttype.org>
Date: Thu Jul 24 17:43:29 2014 +0200

Introduce the double-diamond operator <<>>

This operator works like <> or <ARGV>, as it reads the list of file
names to open from the command-line arguments. However, it disables
the magic-open feature (that forks to execute piped commands) :

$ bleadperl -e 'while(<>){print}' 'echo foo |'
foo
$ bleadperl -e 'while(<<>>){print}' 'echo foo |'
Can't open echo foo |: No such file or directory at -e line 1.

Ricardo Signes

unread,
Jul 24, 2014, 1:13:22 PM7/24/14
to Rafael Garcia-Suarez, perl5-...@perl.org
* Rafael Garcia-Suarez <r...@consttype.org> [2014-07-24T11:51:40]
> I believe that this was discussed here some years ago, and that this
> was suggested by TomC. Anyway, I pushed a patch on rgs/nomagicopen.
> The patch is incomplete as it lacks docs and tests. I welcome feedback
> on the intent of the feature.

Gut reaction after 30s while sitting in a conference session:

I like the feature, but I think it might be too small a change to pick up a new
operator. What about a pragma/feature?

--
rjbs
signature.asc

Kent Fredric

unread,
Jul 24, 2014, 3:27:34 PM7/24/14
to Ricardo Signes, Rafael Garcia-Suarez, perl5-...@perl.org

On 25 July 2014 05:13, Ricardo Signes <perl...@rjbs.manxome.org> wrote:
I like the feature, but I think it might be too small a change to pick up a new
operator.  What about a pragma/feature?

+1 on pragma/featurizing it.

I'm half of the mind that <> allowing pipe opens constitutes a kind of bug in itself.

So a lexical pragma that turns on/off that behavior seems more sensible than new syntax, and once we do that, we can discuss whether or not the pragma should default on/off somehow in a future perl.

Rafael Garcia-Suarez

unread,
Jul 24, 2014, 4:06:11 PM7/24/14
to Ricardo Signes, perl5-...@perl.org
It's more syntactic sugar than new operator. That said, there are benefits
and drawback in the 2 approaches.

Con: The syntax is backward-compatible, so technically we don't *need* a new
pragma to enable it. <<>> optimises for less verbosity.

Pro: However on one-liners without a pragma we lack a way to transform the
implicit while(<>) of -n or -p into a while(<<>>).

Pro: A pragma could also disable the magic in eof() (the one with empty
parentheses).

So what would be a pragma name ? secure::open ? open::nomagic ?

Abigail

unread,
Jul 24, 2014, 4:15:45 PM7/24/14
to Kent Fredric, Ricardo Signes, Rafael Garcia-Suarez, perl5-...@perl.org
-1 here.

People don't like magic open because you can't see what you're getting.

Disabling this with a feature means, you're disabling it with something
you don't see.


while (<>) {
... # Magic open or not?
}


Whether or not it's a magic open is something you can't see locally.
You'd have to scroll the file to see whether a feature is active or not.

while (<<>>) {
... # Obvious
}



Note 1: I'm well aware this is not something specific about <>/<<>>, but
that it's general uneasyness I have with features (or pragmas)
controlling behaviour. It's often too much "action in a distance"
for my liking. (Pragmas also means that whenever you explain something
about a construct, you always have to put a disclaimer there about
the pragmas affecting the construct -- it doesn't make it easier
to learn or teach the language).

Note 2: Having said that, to eliminate the magicness of open(), we got
3-arg open(), instead of controlling it with a pragma.

Note 3: (To ease the feeling of the pumpkin): Instead of seeing '<<>>' as
a new operator, just see it as the '<>' operator with a special
cased operand.


Abigail

Rafael Garcia-Suarez

unread,
Jul 24, 2014, 4:23:18 PM7/24/14
to Eric Brine, perl5-...@perl.org
On 24 July 2014 22:20, Eric Brine <ike...@adaelis.com> wrote:
> The goal is to change the behaviour of reading from ARGV, not <>.

Same thing. <> is exactly equivalent to <ARGV>. Actually the lexer
transforms <> into <ARGV> even before it gets to the parser.

> Unless <<>> is short for <NOTARGV>, this seems like a broken design. What
> happens if someone uses readline(), read() or sysread() on ARGV?

Easy, readline is the same as <>, read and sysread bypass magic
entirely. Nothing new here.

> If <<>> is short for <NOTARGV>, then I think it's a messy solution. Does
> that mean we also have $NOTARGV?

I don't get your point. But note that the open magic is in <>/readline,
not in ARGV. (My patch changes readline, not PL_argvgv.)

> A global pragma sounds much better.

Eric Brine

unread,
Jul 24, 2014, 4:20:04 PM7/24/14
to Rafael Garcia-Suarez, perl5-...@perl.org
The goal is to change the behaviour of reading from ARGV, not <>.

Unless <<>> is short for <NOTARGV>, this seems like a broken design.  What happens if someone uses readline(), read() or sysread() on ARGV?

If <<>> is short for <NOTARGV>, then I think it's a messy solution. Does that mean we also have $NOTARGV?

A global pragma sounds much better.

On Thu, Jul 24, 2014 at 11:51 AM, Rafael Garcia-Suarez <r...@consttype.org> wrote:

Rafael Garcia-Suarez

unread,
Jul 24, 2014, 4:20:29 PM7/24/14
to Abigail, Kent Fredric, Ricardo Signes, perl5-...@perl.org
On 24 July 2014 22:15, Abigail <abi...@abigail.be> wrote:
> On Fri, Jul 25, 2014 at 07:27:34AM +1200, Kent Fredric wrote:
>> On 25 July 2014 05:13, Ricardo Signes <perl...@rjbs.manxome.org> wrote:
>>
>> > I like the feature, but I think it might be too small a change to pick up
>> > a new
>> > operator. What about a pragma/feature?
>> >
>>
>> +1 on pragma/featurizing it.
>>
>> I'm half of the mind that <> allowing pipe opens constitutes a kind of bug
>> in itself.
>>
>> So a lexical pragma that turns on/off that behavior seems more sensible
>> than new syntax, and once we do that, we can discuss whether or not the
>> pragma should default on/off somehow in a future perl.
>
>
> -1 here.
>
> People don't like magic open because you can't see what you're getting.
>
> Disabling this with a feature means, you're disabling it with something
> you don't see.
>
>
> while (<>) {
> ... # Magic open or not?
> }

Ah, yes, I meant to mention that as well.

> Whether or not it's a magic open is something you can't see locally.
> You'd have to scroll the file to see whether a feature is active or not.
>
> while (<<>>) {
> ... # Obvious
> }
>
>
>
> Note 1: I'm well aware this is not something specific about <>/<<>>, but
> that it's general uneasyness I have with features (or pragmas)
> controlling behaviour. It's often too much "action in a distance"
> for my liking. (Pragmas also means that whenever you explain something
> about a construct, you always have to put a disclaimer there about
> the pragmas affecting the construct -- it doesn't make it easier
> to learn or teach the language).
>
> Note 2: Having said that, to eliminate the magicness of open(), we got
> 3-arg open(), instead of controlling it with a pragma.

But then you miss the magicness of the ARGV filehandle.

Eric Brine

unread,
Jul 24, 2014, 10:58:39 PM7/24/14
to Rafael Garcia-Suarez, perl5-...@perl.org
On Thu, Jul 24, 2014 at 4:23 PM, Rafael Garcia-Suarez <r...@consttype.org> wrote:
On 24 July 2014 22:20, Eric Brine <ike...@adaelis.com> wrote:
> The goal is to change the behaviour of reading from ARGV, not <>.

Same thing. <> is exactly equivalent to <ARGV>. Actually the lexer
transforms <> into <ARGV> even before it gets to the parser.

> Unless <<>> is short for <NOTARGV>, this seems like a broken design.  What
> happens if someone uses readline(), read() or sysread() on ARGV?

Easy, readline is the same as <>, read and sysread bypass magic
entirely. Nothing new here. 
 
Then I retract my comment.

demerphq

unread,
Jul 25, 2014, 2:10:17 AM7/25/14
to Rafael Garcia-Suarez, perl5-...@perl.org
+1 on your original implementation.

I am sad that the very first reply to your proposal was a bike-shed comment, and by the pumpking too.

I expect we will bike shed this to death.

Yves


---
perl -Mre=debug -e "/just|another|perl|hacker/"

demerphq

unread,
Jul 25, 2014, 2:13:16 AM7/25/14
to Ricardo Signes, Rafael Garcia-Suarez, perl5-...@perl.org
Unless you have a use for the <<>> operator I dont see your point at all. 

And I am saddened the first reply to Rafaels proposal was a bike-shed comment, and from the pumpking too.

We should just vote up or vote down proposals. No more bike-shedding. Either you have a patch that is an alternative that we can consider or you should vote no and we should let the majority win.

Yves

--

Kent Fredric

unread,
Jul 25, 2014, 2:26:42 AM7/25/14
to demerphq, Ricardo Signes, Rafael Garcia-Suarez, perl5-...@perl.org

On 25 July 2014 18:13, demerphq <deme...@gmail.com> wrote:
We should just vote up or vote down proposals. No more bike-shedding.


+1

I would also welcome a unique thread on this subject so we might (ugh) propose viable anti-bikeshed systems like this one.


Ed Avis

unread,
Jul 25, 2014, 3:00:41 AM7/25/14
to perl5-...@perl.org
Abigail <abigail <at> abigail.be> writes:

>Disabling this with a feature means, you're disabling it with something
>you don't see.
>
> while (<>) {
> ... # Magic open or not?
> }

I would agree, except that 90% of the time someone who programs while (<>)
doesn't consciously intend any kind of magic. It has always been promoted
as the standard way to read files given on the command line. Practically
speaking it is simpler to change it to be safe rather than change the
accumulated weight of tutorial material, existing code and common practice.

(And yes, I know that the Camel does talk about the neat tricks you can do
passing pipe expressions on the command line, but that hasn't really sunk in
as well as it should.)

But, if there is a hidden gotcha that many programmers do not consider,
adding a pragma to turn it off is just about the worst way to fix it. Only
those who already know to take precautions will bother with the pragma.
Perl needs to move towards a state where the default behaviour is safe,
with magic enabled for those who request it.

A new (<<>>) construct doesn't do much to fix existing code, but it doesn't
break it either (for programs where the magic opening is desired and
documented behaviour), and at least it is unambiguous.

--
Ed Avis <e...@waniasset.com>



Ed Avis

unread,
Jul 25, 2014, 3:22:22 AM7/25/14
to perl5-...@perl.org
Just as -E supplements -e to avoid breaking backwards compatibility,
new -N and -P flags could be introduced as safe alternatives to -n and -p.

That said, I suggest the backwards compatibility standards applied to one-
liners may be less strict than those for big programs, so perhaps -n and -p
could just change.

--
Ed Avis <e...@waniasset.com>

Rafael Garcia-Suarez

unread,
Jul 25, 2014, 3:27:40 AM7/25/14
to Ed Avis, perl5-...@perl.org
I think this is actually a good idea.

Abigail

unread,
Jul 25, 2014, 5:44:33 AM7/25/14
to Ed Avis, perl5-...@perl.org
I'd say that if there's one place where magic open is useful, it's
with -n and -p.

Considering that with one liners, you're typically in full control of
what you type, including the command line arguments, I'm not sure
whether just changing -n and -p, robbing the user of options, is the
right way.


When was the last time you couldn't write a -n/-p one liner, because
one of the arguments you wanted to give would trigger magic open, and
you didn't want that to happen?



Abigail

Ricardo Signes

unread,
Jul 26, 2014, 6:38:03 PM7/26/14
to demerphq, Rafael Garcia-Suarez, perl5-...@perl.org
* demerphq <deme...@gmail.com> [2014-07-25T02:13:16]
> Unless you have a use for the <<>> operator I dont see your point at all.

Not every new behavior needs to be implemented as an operator. There are
reasons to use or not use an operator for any given behavior. It's not the
case that we use an operator whenever somebody does a first-pass implementation
that way and we can't immediately propose some other use for the operator.
Instead, if we have a suggested new behavior, we can consider the behavior
apart from the question of whether or not it's been exposed in the right way.

I also think that in most cases, we have quite a lot of generic features that
can be used for new behaviors, and making a new kind of syntax is something
that has to provide a good reason. So, the question is: is there one, here?

I'll discuss that in another post, as others engaged the question and I'd like
to reply to that. I'm writing this post mostly because of the following:

> And I am saddened the first reply to Rafaels proposal was a bike-shed
> comment, and from the pumpking too.
>
> We should just vote up or vote down proposals. No more bike-shedding.
> Either you have a patch that is an alternative that we can consider or you
> should vote no and we should let the majority win.

I think this is a very bad idea.

Under this regime, someone could supply a patch that's a cool feature. Then,
someone else could notice a significant problem with it, but without C skills
to implement a solution, they would be disallowed from even pointing out the
problem. That's a complete non-starter.

Language changes are things that have to be designed, and that means
discussion.

Here's my counter-proposal on bike-shedding: no more using the phrase
bike-shedding. It's now used to mean "somebody wants to talk about something
where I think I'm already right," and it's too emotionally charged. If you
think someone is debating pointless trivia, suggesting changes of no
consequence, or can't see the forest for the trees, point out the specific
problematic behavior.

There is absolutely no way on Earth that I am going along with "apply or don't
apply, but never discuss," though.

--
rjbs
signature.asc

Ricardo Signes

unread,
Jul 26, 2014, 7:16:21 PM7/26/14
to Rafael Garcia-Suarez, perl5-...@perl.org
I'm quoting Rafael slightly out of order!

* Rafael Garcia-Suarez <r...@consttype.org> [2014-07-24T16:06:11]
> It's more syntactic sugar than new operator. That said, there are benefits
> and drawback in the 2 approaches.

I think you did a good job hitting the nail on the head with the same things
I'd thought about this. (Plus eof(), which I had not considered!)

> So what would be a pragma name ? secure::open ? open::nomagic ?

I'm not too worried about the name yet. Of those two, I prefer 'open::nomagic'
because "no magic" is a bit safer of a claim than "secure." :) And we'll want
to be clear whether this is affecting "open" or "open when dealing with ARGV".
Let's come back to it, though? I'm more concerned about whether syntax or a
pragma will pay off better in the long run.

> Con: The syntax is backward-compatible, so technically we don't *need* a new
> pragma to enable it. <<>> optimises for less verbosity.
>
> Pro: However on one-liners without a pragma we lack a way to transform the
> implicit while(<>) of -n or -p into a while(<<>>).

I think Abigail was right to note that we have to be wary of spooky action at a
distance. On the other hand, I think that what you've given us is much more
useful as a fixed default than an optional new behavior. That is: in *most*
circumstances, I think the behavior of <> given @ARGV of ("rm -rfv *|") is
going to surprise. Sure, it's documented, but that's not always a good reason
to expect people to have the right expectations. :)
(Also documented: ARGV::readonly -- but does anybody use it?)

So, anyway, there are three reasons to want syntax for this:

* there's no question about what "kind" of <> is active
* no pragma/feature is needed to make it available
* you can use both in one program (although I think this is a stretch! 😄 )

...and some reasons to avoid it:

* the programmer needs to remember to use the new form to avoid magic
* it doesn't help with -n etc.

Abigail notes that command-line-switch-generated code is often a place where we
*do* want the magic. So do we want a way to get -n-but-with-nomagic-open? Ed
and Rafael both seemed to like -P-and-so-on.

To me, the most significant concern is probably the one about improving our
defaults. I'm not sure I think this is the best solution, but let me know if
you think it's clearly better or worse, or if it suggests anything else to you:

* use feature 'magic_open_argv'
* ...which goes into the default feature bundle, and old versions
* ...but not the new version's bundle, so it's off by default on use 5.22
* ...and it's on in the while() code generated by -n
* ...but we have -N and -P

Although in my fantasy world, I'd make -n and -p *not* use magic open argv, I
think we're liable to break too many familiar one-liners. I also think we'll
better off making sure that the behavior of -p/-P is always *explicit*, rather
than tied to -E/-e.

Like I said: do you think this is better, worse, or just different?

--
rjbs
signature.asc

Smylers

unread,
Jul 27, 2014, 3:21:22 AM7/27/14
to perl5-...@perl.org
Ricardo Signes writes:

> To me, the most significant concern is probably the one about
> improving our defaults.

I think so too, though I'm mildly concerned about The Blub Paradox:
http://www.paulgraham.com/avg.html — I can't remember ever passing a
pipe as an argument to a Perl program, but perhaps that's because I've
been missing out on a useful feature. Or by using magic <> I've been
writing handy generic programs which other users can usefully pass pipes
to, and by removing the magic I'd also be removing a useful feature.

As such, it'd be good to hear from somebody who has used this feature
(whether successfully or disastrously).

> * use feature 'magic_open_argv'
> * ...which goes into the default feature bundle, and old versions
> * ...but not the new version's bundle, so it's off by default on use 5.22

That sounds good: safe by default, but anybody who knows they want the
magic can easily opt in. In particular, it means that the beginner
simply doesn't need to know about any of it.

> * ...and it's on in the while() code generated by -n

Even if the one-liner also supplies -Mv5.22? That could be confusing.

> * ...but we have -N and -P

I'm less persuaded of the need for these, because they require opting in
to safety and because of Abigail's point about a one-liner author being
in control of arguments. But they do make Perl more complicated — more
options in perlrun to have to learn.

And anybody giving examples of one-liners — whether in a Perl tutorial,
or a command for a sys-admin to run (and which just happens to be in
Perl) — has to continue to use the ‘unsafe’ -n and -p anyway for
backwards compatibility, or uses -N and -P and has readers being
frustrated with Perl because it doesn't work in their version, or has to
mention both with an explanation of which to use. None of those sound
like an improvement to me.

> I also think we'll better off making sure that the behavior of -p/-P
> is always *explicit*, rather than tied to -E/-e.

I think that introducing exceptions to what -E does would also be
unfortunate, in terms of adding to the cognitive load of learning Perl.

However, I note that -E is currently documented in two places, that
don't entirely agree.

Quick poll, before reading on past the following space: What do you
think -E does?
























Here are the two definitions:

• The feature docs say that -E enables the feature bundle for the
version of Perl being used. So on perl 5.22, -E should be the same as
use v5.22.

• perlrun says that -E enables all optional features. So on perl5.22 -E
should enable magic_open_argv even though it isn't in the bundle for
use v5.22.

Those two definitions are only resolve to the same features as long as
newer versions of Perl only add to the set of features in the default
feature bundle. Which is currently true. But this proposal is (sensibly,
it seems to me) is to remove a feature.

Whatever we decide about the behaviour of -n and -p, we should decide on
what the definition of -E actually is and fix the docs. I can see
benefits to either approach: both ‘make this behave the default way for
this version of Perl’, and ‘just give me all the features, without me
having to type out long -M lines to enable them’ could be useful.

I think the least-confusing approach would be to go with whatever most
users already think the definition for -E is.

Smylers
--
Girls don't wear their shoes out, and boys don't care about style?
Clarks think so: http://j.mp/clarksgirlboyposters
Disagree? Don't want shops promoting gender stereotypes to children?
Please sign the petition: http://j.mp/clarksgirlboypetition

Rafael Garcia-Suarez

unread,
Jul 27, 2014, 3:19:36 AM7/27/14
to Ricardo Signes, perl5-...@perl.org
Why the double negation? use feature 'nomagic_open_argv' should
work similarly.

Dave Mitchell

unread,
Jul 27, 2014, 3:23:47 AM7/27/14
to bulk 88, Ricardo Signes, demerphq, Rafael Garcia-Suarez, perl5-...@perl.org
On Sat, Jul 26, 2014 at 11:42:06PM -0400, bulk 88 wrote:
> The davem grant reports have hours
> of p5p reading on them. If that is what he does, how many non-paid
> "volunteers" are spending an equal number of hours reading p5p
> ML instead of debugging/writing code?

My grant reports say "processing" my p5p mailbox, not "reading" it. So it
covers all the stuff I do *while* reading - e.g.. looking at issues,
having a look at some source code, running a bisect etc - where the work
isn't big enough to get its own separate entry on my timesheet.

--
Counsellor Troi states something other than the blindingly obvious.
-- Things That Never Happen in "Star Trek" #16

Dave Mitchell

unread,
Jul 27, 2014, 3:32:17 AM7/27/14
to bulk 88, Ricardo Signes, demerphq, Rafael Garcia-Suarez, perl5-...@perl.org
On Sat, Jul 26, 2014 at 11:42:06PM -0400, bulk 88 wrote:
> A number of people (not naming names) who go into language design
> threads, I never see their names except in language design.
> They do not otherwise participate in P5P, no tickets, no ticket comments,
> no patches.

There is nothing wrong with that. Some people have good language design
skills, while others have good implementation skills. Both can bring
something to the discussion. I know personally that I'm rubbish at
language design, and I'm grateful if someone can tell me "that won't work
because X is already valid syntax"; conversely I can tell language
designers that their pet idea is unimplementable.

Think interior designer verses builder/decorator.

--
Overhead, without any fuss, the stars were going out.
-- Arthur C Clarke

Smylers

unread,
Jul 27, 2014, 5:03:56 AM7/27/14
to perl5-...@perl.org
This way, the only double negative is in Rik's description of the plan.
From a Perl programmer's point of view, it's simply that somebody
wanting to use the feature use-s it.

> use feature 'nomagic_open_argv' should work similarly.

But that imposes the double negative on users. In that if in perl 5.22
nomagic_open_argv is the default, somebody wanting the magic behaviour
would have to do:

no feature qw<nomagic_open_argv>;

That's unquestionably a double negative, and is far harder to grok than:

use feature qw<magic_open_argv>;

It also prevents bundling all the features you want into a single call.
Instead of:

use feature qw<say magic_open_argv unicode_strings signatures>;

You'd have to write:

use feature qw<say unicode_strings signatures>;
no feature qw<nomagic_open_argv>;

It'd be a weird exception that most features are enabled positively but
one of them is enabled by disabling its opposite — an exception that's
only explained by historical reasons.

So I think nomagic_open_argv is much worse from the point of view of not
avoiding banishing double negatives ...

Ricardo Signes

unread,
Jul 27, 2014, 9:31:09 AM7/27/14
to perl5-...@perl.org
* Smylers <Smy...@stripey.com> [2014-07-27T05:03:56]
> Rafael Garcia-Suarez writes:
> > Why the double negation?
>
> This way, the only double negative is in Rik's description of the plan.
> From a Perl programmer's point of view, it's simply that somebody
> wanting to use the feature use-s it.
>
> > use feature 'nomagic_open_argv' should work similarly.
>
> But that imposes the double negative on users. In that if in perl 5.22
> nomagic_open_argv is the default, somebody wanting the magic behaviour
> would have to do:
>
> no feature qw<nomagic_open_argv>;
> [...]

Smylers has communicated my thoughts for me. :)

--
rjbs
signature.asc

Peter Martini

unread,
Jul 27, 2014, 9:50:59 AM7/27/14
to Abigail, Ed Avis, perl5-...@perl.org
On 7/25/14, Abigail <abi...@abigail.be> wrote:
>
> Considering that with one liners, you're typically in full control of
> what you type, including the command line arguments, I'm not sure
> whether just changing -n and -p, robbing the user of options, is the
> right way.
>
>
> When was the last time you couldn't write a -n/-p one liner, because
> one of the arguments you wanted to give would trigger magic open, and
> you didn't want that to happen?

It's actually more dangerous and has sharper edges than may be
initially considered. Putting on my security hat:

shell $ touch 'report6; ls -l * |'
shell $ perl -pe '' report*

Even with one liners, the user isn't necessarily aware of the
arguments that will be processed.

Peter Martini

unread,
Jul 27, 2014, 10:11:12 AM7/27/14
to Rafael Garcia-Suarez, perl5-...@perl.org
I definitely like the idea in some form, though I'll sit out the
feature/syntax discussion for the moment. I want to clarify the
framing of the feature though:

When you say 'disable magic-open', a more precise description might be
'Changes ARGV handling to use 3 arg open instead of 2 arg open',
correct? That way its a little more clearly tied to 2 arg / 3 arg
open discussions and makes it easier to spot the edge cases.

Phrasing it that way makes it a little clearer where the magic comes
from and what other things would be affected; for instance (I can't
test this right now)

perl -e 'print while(<<>>)' '-'

would now only work on a file literally named '-'?

Rafael Garcia-Suarez

unread,
Jul 27, 2014, 10:19:30 AM7/27/14
to Peter Martini, perl5-...@perl.org
Yes. However, an empty ARGV would still default to reading STDIN.

Abigail

unread,
Jul 27, 2014, 1:55:23 PM7/27/14
to Peter Martini, Ed Avis, perl5-...@perl.org
Sure, but even with magic open disabled,

$ perl -pe '' *

isn't safe, as someone may have created a file starting with -e.


It's "not knowing what your wildcard expands to" which is the dangerous
thing. Magic open is just one of the thing wildcard expansion can make
use of. But disabling magic open doesn't make wildcard expansion safe
if you don't know what it expands to.



Abigail

Abigail

unread,
Jul 27, 2014, 2:19:44 PM7/27/14
to Ricardo Signes, demerphq, Rafael Garcia-Suarez, perl5-...@perl.org
On Sat, Jul 26, 2014 at 06:38:03PM -0400, Ricardo Signes wrote:
>
> Here's my counter-proposal on bike-shedding: no more using the phrase
> bike-shedding. It's now used to mean "somebody wants to talk about something
> where I think I'm already right," and it's too emotionally charged. If you
> think someone is debating pointless trivia, suggesting changes of no
> consequence, or can't see the forest for the trees, point out the specific
> problematic behavior.


Hear, hear.

It always irks me if people complain that there are a lot of comments
how a proposed feature looks, and far less on how it's implemented.
Or that non-visual changes hardly get comments at all.

But guess what. How something looks actually matters to people. A lot.
You can design a perfect car, but if it looks ugly, it won't sell
very well. And that's also why a lot of big companies A/B test things
on their website that are nothing but appearance. Because it matters.
(And physical products are often tested for their appearance in panels
as well).

And if one goes outside of the echo chamber where everything Perl
is praised, what's the biggest complaint about Perl? Not that it's
implemented badly. But that it looks ugly.

How things look matters to Larry as well. To give an example, he once
told that the fact that "last", "next" and 'redo" all use four characters
isn't a coincidence, but it was a deliberate choice. It wouldn't make
the implementation any easier or harder had, say, "redo" be named
"again". But appearance mattered.

Back to the feature itself. I prefer '<<>>' over using a feature. It's
clearer (because you can see what's happening), and, since we're talking
safety, safer. Imagine:


use 5.022; # Enables turning off magic-open.

...

while (<>) {
...
}


Time passes, and the code gets run [1] on 5.20. It dies with a version
error. Someone removes the 'use 5.022' (what, you've never done that?
I have), and lo, it works. Well, with the safety off.


[1] Code gets moved around, copied, reused, etc.



Abigail

Ricardo Signes

unread,
Jul 27, 2014, 6:10:20 PM7/27/14
to Abigail, demerphq, Rafael Garcia-Suarez, perl5-...@perl.org
* Abigail <abi...@abigail.be> [2014-07-27T14:19:44]
> Imagine:
>
> use 5.022; # Enables turning off magic-open.
> ...
> while (<>) {
> ...
> }
>
> Time passes, and the code gets run [1] on 5.20. It dies with a version
> error. Someone removes the 'use 5.022' (what, you've never done that?
> I have), and lo, it works. Well, with the safety off.

Yeah, this is precisely what I've been frowning at all day. I'm still not a
huge fan of <<>>, but I think in the end, it will be better than features. I'm
not decided either way, I'm flip-flopping around, but I've always been pretty
unhappy about invisible changes being introduced by features (because then you
can't get rid of the feature, ever, for one thing!), and I think this is
probably another place where it would be a problem.

What I *didn't* say originally was that this would be a great place for adverbs
in the Perl 6 style. I'd hardly say that we should not merge <<>> just because
adverbs would be better, but it would be a *resuable* and *multi-purpose*
extension of syntax. For example:

while (<>:nomagic) {
...
}

So maybe this message is a formal notice of "this would be useful here," so
that I remember it in the future in other cases, like <>:chomp and whatever
else comes to mind.

--
rjbs
signature.asc

Aristotle Pagaltzis

unread,
Jul 27, 2014, 6:44:31 PM7/27/14
to perl5-...@perl.org
* Abigail <abi...@abigail.be> [2014-07-27 20:00]:
> On Sun, Jul 27, 2014 at 09:50:59AM -0400, Peter Martini wrote:
> > On 7/25/14, Abigail <abi...@abigail.be> wrote:
> > >
> > > Considering that with one liners, you're typically in full control of
> > > what you type, including the command line arguments, I'm not sure
> > > whether just changing -n and -p, robbing the user of options, is the
> > > right way.
> > >
> > >
> > > When was the last time you couldn't write a -n/-p one liner, because
> > > one of the arguments you wanted to give would trigger magic open, and
> > > you didn't want that to happen?
> >
> > It's actually more dangerous and has sharper edges than may be
> > initially considered. Putting on my security hat:
> >
> > shell $ touch 'report6; ls -l * |'
> > shell $ perl -pe '' report*
> >
> > Even with one liners, the user isn't necessarily aware of the
> > arguments that will be processed.
>
> Sure, but even with magic open disabled,
>
> $ perl -pe '' *
>
> isn't safe, as someone may have created a file starting with -e.

$ cd `mktemp -d ./scratch-XXXXXXXXXXXX`
$ echo 'All clear, move along.' > test
$ touch -- '-edie'
$ perl -pe '' *
Died at -e line 2, <> line 1.

You are correct: that is unsafe.

> It's "not knowing what your wildcard expands to" which is the
> dangerous thing. Magic open is just one of the thing wildcard
> expansion can make use of. But disabling magic open doesn't make
> wildcard expansion safe if you don't know what it expands to.

$ perl -pe '' -- *
All clear, move along.

You are incorrect: that is not unsafe.

Not knowing what your wildcard expands to is not the dangerous thing.

Not knowing that wildcard expansion in general can have a semi-predicate
problem – and then not addressing it – is the dangerous thing.

You are of course dependent on your programs and scripts to provide some
way (such as the -- argument convention) to resolve the ambiguity. But
perl does, so you can.

Even if you have no idea what your wildcard is going to expand to.

But with the diamond operator (and constructs which implicitly use it),
you cannot. There is no switch you can pass or alternative you can use
to say “I want all of these treated as filenames, regardless of what
they look like”, like you can with filenames vs switches. You have no
way to avoid the semi-predicate problem of the diamond operator other
than avoiding its use entirely.

Regards,
--
Aristotle Pagaltzis // <http://plasmasturm.org/>

Uri Guttman

unread,
Jul 27, 2014, 7:00:23 PM7/27/14
to perl5-...@perl.org
On 07/27/2014 06:44 PM, Aristotle Pagaltzis wrote:
> * Abigail <abi...@abigail.be> [2014-07-27 20:00]:


>
> But with the diamond operator (and constructs which implicitly use it),
> you cannot. There is no switch you can pass or alternative you can use
> to say “I want all of these treated as filenames, regardless of what
> they look like”, like you can with filenames vs switches. You have no
> way to avoid the semi-predicate problem of the diamond operator other
> than avoiding its use entirely.

at oscon last week, damian showed a perl6 (and perl5 version) of a
rename script. the example one liner he showed had * in it but he did
globbing on that inside the script. afterwards both merlyn and myself
asked the same question, why didn't the shell glob first? damian said he
put a noglob command in an alias to call his script. not saying this is
a solution to no magic but it doesn't force the arguments (after
options) to be files and not globbed for you.

uri


--
Uri Guttman - The Perl Hunter
The Best Perl Jobs, The Best Perl Hackers
http://PerlHunter.com

Ed Avis

unread,
Jul 27, 2014, 10:06:06 PM7/27/14
to perl5-...@perl.org
...and now that you mention the magic filename - I begin to think that -
should still mean standard input, even though >foo does not mean to weirdly
overwrite a file called foo. The expectations around this are subtle...

In a way having - for stdin is more reasonable because arguments beginning
with that character are already 'reserved' as options (for all that the shell
wildcard expansion ignores this convention).

But I wouldn't want to let quibbles over - get in the way of the more
important fix of eliminating the unsafe behaviour on <>|.

--
Ed Avis <e...@waniasset.com>

Abigail

unread,
Jul 28, 2014, 5:02:24 AM7/28/14
to Aristotle Pagaltzis, perl5-...@perl.org
But that's what's being proposed, isn't it? A new
operator/feature/command-line switch. I'm not at all against the *option*
to disable magic open. Just like you have the *option* to stop processing
options by using '--'.

I've nothing against -N/-P for non-magic open. But that's something else
than changing -n/-p to eliminate magic open. Besides, it gives a tricky
form of security. "perl -ne '...' -- *" may be 'safe' from magic open
on perl 5.22, but it's still unsafe if you run this command on a box
that just hasn't upgraded its perl yet.



Abigail

Ricardo Signes

unread,
Jul 29, 2014, 5:54:24 PM7/29/14
to Rafael Garcia-Suarez, perl5-...@perl.org
* Rafael Garcia-Suarez <r...@consttype.org> [2014-07-24T11:51:40]
> I believe that this was discussed here some years ago, and that this
> was suggested by TomC. Anyway, I pushed a patch on rgs/nomagicopen.
> The patch is incomplete as it lacks docs and tests. I welcome feedback
> on the intent of the feature.

I'm still not crazy about the one-off syntax addition, but I think that it's
the best we're going to see without setting an unrealistic bar for a new,
generic feature. If we ever *do* get <> adverbs, we can call this sugar for
one or blah blah blah it doesn't matter that much right now. :-)

While my heart wants -n to be safe, my conscience tells me that we'll break
enough (bizarre, to my mind (except maybe regarding "-")) expectations that it
isn't worth doing. Probably we want -P and -N for safe opening.

I am gratified to see that <<X>> fails for any X, as you've only made the
literal construct <<>> iterate. Awesome. I think we're going to want a better
error message, if possible, though, than:

~/code/perl5$ ./perl -e 'while (<<ARGV>>) { print }' 'ls |'
Can't find string terminator "ARGV" anywhere before EOF at -e line 1.

Something roughly like "safe <<>> operator meaningless on concrete filehandles"
— that's not right, but you get the idea.

I don't think I have any further thoughts on it.

--
rjbs
signature.asc

Abigail

unread,
Jul 29, 2014, 6:34:27 PM7/29/14
to Ricardo Signes, Rafael Garcia-Suarez, perl5-...@perl.org
That may be hard, as this:

while (<<ARGV>>3) {print}
ARGV

is legal syntax.

You'd need to look ahead a couple of tokens to determine whether '<<ARGV>>'
is reading from a filehandle, or shifting a here doc.


Abigail

Ricardo Signes

unread,
Jul 29, 2014, 8:04:03 PM7/29/14
to Abigail, Rafael Garcia-Suarez, perl5-...@perl.org
* Abigail <abi...@abigail.be> [2014-07-29T18:34:27]
> On Tue, Jul 29, 2014 at 05:54:24PM -0400, Ricardo Signes wrote:
> >
> > I am gratified to see that <<X>> fails for any X, as you've only made the
> > literal construct <<>> iterate. Awesome. I think we're going to want a
> > better error message, if possible, though, than:
> >
> > ~/code/perl5$ ./perl -e 'while (<<ARGV>>) { print }' 'ls |'
> > Can't find string terminator "ARGV" anywhere before EOF at -e line 1.
>
> That may be hard, as this:
>
> while (<<ARGV>>3) {print}
> ARGV
>
> is legal syntax.

Yeah, the error that I got made it clear that it would be tricky. If we can't,
we can't, but if we can, we should. (For non-absurd-effort levels of "can".)

--
rjbs
signature.asc

David Nicol

unread,
Jul 29, 2014, 9:07:44 PM7/29/14
to Ricardo Signes, Abigail, Rafael Garcia-Suarez, perl5-...@perl.org
>> > literal construct <<>> iterate. Awesome. I think we're going to want a
>> > better error message, if possible, though, than:
>> >
>> > ~/code/perl5$ ./perl -e 'while (<<ARGV>>) { print }' 'ls |'
>> > Can't find string terminator "ARGV" anywhere before EOF at -e line 1.
>>
>> That may be hard, as this:
>>
>> while (<<ARGV>>3) {print}
>> ARGV
>>
>> is legal syntax.
>
> Yeah, the error that I got made it clear that it would be tricky. If we can't,
> we can't, but if we can, we should. (For non-absurd-effort levels of "can".)

somewhere in this archive is a patch I submitted once to allow
whitespace between a
left-shift that begins an expression and a bareword that isn't an
infix operator (to allow
using the deprecated empty terminator, which is what that always
confusingly misparses as,
to precede C<le> for instance) and that might be a good starting place

Rafael Garcia-Suarez

unread,
Jul 30, 2014, 11:13:43 AM7/30/14
to Ricardo Signes, perl5-...@perl.org
On 29 July 2014 23:54, Ricardo Signes <perl...@rjbs.manxome.org> wrote:
> * Rafael Garcia-Suarez <r...@consttype.org> [2014-07-24T11:51:40]
>> I believe that this was discussed here some years ago, and that this
>> was suggested by TomC. Anyway, I pushed a patch on rgs/nomagicopen.
>> The patch is incomplete as it lacks docs and tests. I welcome feedback
>> on the intent of the feature.
>
> I'm still not crazy about the one-off syntax addition, but I think that it's
> the best we're going to see without setting an unrealistic bar for a new,
> generic feature. If we ever *do* get <> adverbs, we can call this sugar for
> one or blah blah blah it doesn't matter that much right now. :-)
>
> While my heart wants -n to be safe, my conscience tells me that we'll break
> enough (bizarre, to my mind (except maybe regarding "-")) expectations that it
> isn't worth doing. Probably we want -P and -N for safe opening.
>
> I am gratified to see that <<X>> fails for any X, as you've only made the
> literal construct <<>> iterate. Awesome. I think we're going to want a better
> error message, if possible, though, than:
>
> ~/code/perl5$ ./perl -e 'while (<<ARGV>>) { print }' 'ls |'
> Can't find string terminator "ARGV" anywhere before EOF at -e line 1.

As Abigail pointed out that might be a bit difficult to implement.
Unless we write it «» instead :)

Eric Brine

unread,
Jul 31, 2014, 10:46:06 AM7/31/14
to Rafael Garcia-Suarez, Ricardo Signes, perl5-...@perl.org
On Wed, Jul 30, 2014 at 11:13 AM, Rafael Garcia-Suarez <r...@consttype.org> wrote:
As Abigail pointed out that might be a bit difficult to implement.
Unless we write it «» instead :)

Not hard, just backwards-incompatible and introduces a grammar ambiguity.

Have you ever seen anyone bit shift a heredoc?

... <<NUM >> 3 ...
4
NUM

And as ambiguities in Perl go, this would probably the mildest one.

David Nicol

unread,
Jul 31, 2014, 10:54:42 AM7/31/14
to Eric Brine, Perl 5 Porters

Eric Brine may have missed the detail that « and » are single characters that are currently unused. Abigail's suggestion referred to the prospect of finally, after years of patience, hearing the hiss as the seal is pierced on the vacuum packed worm can labeled, IIRC, "unicode operators."

On Thu, Jul 31, 2014 at 9:46 AM, Eric Brine <ike...@adaelis.com> wrote:
> Unless we write it «» instead :)
Have you ever seen anyone bit shift a heredoc?

Captain Obvious at your service,
dln

Eric Brine

unread,
Jul 31, 2014, 12:29:58 PM7/31/14
to David Nicol, Perl 5 Porters
On Thu, Jul 31, 2014 at 10:54 AM, David Nicol <david...@gmail.com> wrote:

Eric Brine may have missed the detail that « and » are single characters that are currently unused.

No, you missed the point that there's no need to use characters that aren't found on a hemisphere's keyboards.


bulk88

unread,
Jul 31, 2014, 12:57:29 PM7/31/14
to Eric Brine, David Nicol, Perl 5 Porters
Is this whole subthread about unicode characters being operators really
needed? Nobody saw the smiley in RGS's post
http://www.nntp.perl.org/group/perl.perl5.porters/2014/07/msg218271.html ?

Rafael Garcia-Suarez

unread,
Sep 30, 2014, 2:33:41 AM9/30/14
to Ricardo Signes, perl5-...@perl.org
On 29 July 2014 23:54, Ricardo Signes <perl...@rjbs.manxome.org> wrote:
I have now merged this in blead, with more tests, and docs by Peter Martini
and myself. I also added tests for the $ARGV variable, that was not tested.

I'm not too sure about the error message; it's not totally trivial to
change, since
a script like this one, for example, will print "8":

print <<FOO>> 1;
16
FOO

Father Chrysostomos

unread,
Sep 30, 2014, 9:05:49 AM9/30/14
to perl5-...@perl.org
Rafael Garcia-Suarez wrote:
> I have now merged this in blead, with more tests, and docs by Peter Martini
> and myself. I also added tests for the $ARGV variable, that was not tested.

It looks as though you have changed the behaviour of eof(), though I
have not tried building it yet.

Would it not be better to flag in GvEGVx(PL_argvgv) to remember which
of <>/<<>> was last used? There is plenty of room in GvFLAGS.

Rafael Garcia-Suarez

unread,
Sep 30, 2014, 10:00:02 AM9/30/14
to perl5-...@perl.org
On 30 September 2014 15:05, Father Chrysostomos <spr...@cpan.org> wrote:
> Rafael Garcia-Suarez wrote:
>> I have now merged this in blead, with more tests, and docs by Peter Martini
>> and myself. I also added tests for the $ARGV variable, that was not tested.
>
> It looks as though you have changed the behaviour of eof(), though I
> have not tried building it yet.

Maybe, but I don't see where? By the way, I just pushed a test
involving eof and <<>> as ad77c200c8a4ed. eof() (with empty parentheses)
still continues to open the next file the old way.

> Would it not be better to flag in GvEGVx(PL_argvgv) to remember which
> of <>/<<>> was last used? There is plenty of room in GvFLAGS.

What would that solve? The point is that <> and <<>> differ on how they
open the next upcoming file. This is not a property of ARGV.

Father Chrysostomos

unread,
Sep 30, 2014, 12:17:14 PM9/30/14
to perl5-...@perl.org
Rafael Garcia-Suarez wrote:
> > It looks as though you have changed the behaviour of eof(), though I
> > have not tried building it yet.
>
> Maybe, but I don't see where?

OK, I think I misread the code. I guess it's the double negative of
'nomagicopen' being set to FALSE that confused me.

> By the way, I just pushed a test
> involving eof and <<>> as ad77c200c8a4ed.

I don't see eof() in that commit, though the test you added is
a good one.

> eof() (with empty parentheses)
> still continues to open the next file the old way.
>
> > Would it not be better to flag in GvEGVx(PL_argvgv) to remember which
> > of <>/<<>> was last used? There is plenty of room in GvFLAGS.
>
> What would that solve? The point is that <> and <<>> differ on how they
> open the next upcoming file. This is not a property of ARGV.

If I understand correctly, eof() tries opening the next file at an
individual file's EOF. I thought you might consider having it reuse
the last <> or <<>>, to avoid giving a false sense of security to peo-
ple who do:

while (<<>>) {
...
blah_blah_blah() if eof();
}

where the eof() might to a 2-arg open.

Then again, I haven't tested any of this (I don't have time right
now), so I may be completely wrong.
..

Rafael Garcia-Suarez

unread,
Sep 30, 2014, 4:44:15 PM9/30/14
to perl5-...@perl.org
On 30 September 2014 18:17, Father Chrysostomos <spr...@cpan.org> wrote:
> Rafael Garcia-Suarez wrote:
>> > It looks as though you have changed the behaviour of eof(), though I
>> > have not tried building it yet.
>>
>> Maybe, but I don't see where?
>
> OK, I think I misread the code. I guess it's the double negative of
> 'nomagicopen' being set to FALSE that confused me.
>
>> By the way, I just pushed a test
>> involving eof and <<>> as ad77c200c8a4ed.
>
> I don't see eof() in that commit, though the test you added is
> a good one.

Not explicitly, but I wanted to test that opening the next file
was DTRT. (Also, so many tests missing...)

>> eof() (with empty parentheses)
>> still continues to open the next file the old way.
>>
>> > Would it not be better to flag in GvEGVx(PL_argvgv) to remember which
>> > of <>/<<>> was last used? There is plenty of room in GvFLAGS.
>>
>> What would that solve? The point is that <> and <<>> differ on how they
>> open the next upcoming file. This is not a property of ARGV.
>
> If I understand correctly, eof() tries opening the next file at an
> individual file's EOF. I thought you might consider having it reuse
> the last <> or <<>>, to avoid giving a false sense of security to peo-
> ple who do:
>
> while (<<>>) {
> ...
> blah_blah_blah() if eof();
> }
>
> where the eof() might to a 2-arg open.

This is a good point, and <<>> does not solve this problem. I should
at least make that explicit in the docs. But it would be better if we
could just "stick" the nomagicopen flag to eof(). (I hate the eof()/eof
distinction and avoid using those for that reason)

Michael Felt

unread,
Oct 3, 2014, 6:27:38 AM10/3/14
to Rafael Garcia-Suarez, Ricardo Signes, perl5-...@perl.org

this may have nothing to do with this thread, other than it is the only one search returned. I see a lot of argv.t test failures on my AIX smokers. fyi.

Rafael Garcia-Suarez

unread,
Oct 3, 2014, 7:44:32 AM10/3/14
to Michael Felt, Ricardo Signes, perl5-...@perl.org
Hi, I see the failures on the reports now:

../t/io/argv.t .............................................. FAILED 29-32

Could you run the test by hand and send the output?

H.Merijn Brand

unread,
Oct 3, 2014, 7:59:12 AM10/3/14
to perl5-...@perl.org
On Fri, 3 Oct 2014 13:44:32 +0200, Rafael Garcia-Suarez
<r...@consttype.org> wrote:

> Hi, I see the failures on the reports now:
>
> ../t/io/argv.t .............................................. FAILED 29-32
>
> Could you run the test by hand and send the output?

$ ./perl -I../lib harness -v io/argv.t
io/argv.t ..
1..36
ok 1 - <> from two files
ok 2 - <> from a file and STDIN
ok 3 - <> from just STDIN
ok 4 - $ARGV is the file name
ok 5 - $ARGV is - for explicit STDIN
ok 6 - $ARGV is - for implicit STDIN
ok 7 - eof() doesn't segfault
ok 8 - $. counts <>
ok 9 - <> from @ARGV
ok 10
ok 11
ok 12
ok 13 - eof() true on unopened filehandle
ok 14 - STDIN has something
ok 15
ok 16 - eof() true with empty @ARGV
ok 17
ok 18
ok 19 - eof() true after closing ARGV
ok 20
ok 21
ok 22
ok 23
ok 24
ok 25
ok 26 - <<>>
ok 27 - <<>> and rcatline
ok 28 - <<>> from just STDIN (no argument)
ok 29 - $ARGV is - for STDIN with <<>>
not ok 30 - <<>> does not treat - as STDIN
# Failed test 30 - <<>> does not treat - as STDIN at io/argv.t line 191
# got "Can\'t open -: A file or directory in the path name does not exist. at -e line 1.\n"
# expected "Can\'t open -: No such file or directory at -e line 1.\n"
not ok 31 - <<>> does not treat - as STDIN
# Failed test 31 - <<>> does not treat - as STDIN at io/argv.t line 199
# got "Can\'t open : A file or directory in the path name does not exist. at -e line 1.\n"
# expected "Can\'t open : No such file or directory at -e line 1.\n"
not ok 32 - <<>> does not treat - as STDIN
# Failed test 32 - <<>> does not treat - as STDIN at io/argv.t line 205
# got "Can\'t open : A file or directory in the path name does not exist. at -e line 1.\n"
# expected "Can\'t open : No such file or directory at -e line 1.\n"
not ok 33 - <<>> does not treat ...| as fork
# Failed test 33 - <<>> does not treat ...| as fork at io/argv.t line 216
# got "Can\'t open echo foo |: A file or directory in the path name does not exist. at -e line 1.\n"
# expected "Can\'t open echo foo |: No such file or directory at -e line 1.\n"
# Failed test 34 - <<>> does not treat ...| as fork after eof at io/argv.t line 223
# got "Can\'t open echo foo |: A file or directory in the path name does not exist. at -e line 1, <> line 3.\n"
# expected "Can\'t open echo foo |: No such file or directory at -e line 1, <> line 3.\n"
not ok 34 - <<>> does not treat ...| as fork after eof
ok 35 - ARGV aliasing and eof()
ok 36 - deleting $::{ARGV}
Failed 5/36 subtests

Test Summary Report
-------------------
io/argv.t (Wstat: 0 Tests: 36 Failed: 5)
Failed tests: 30-34
Files=1, Tests=36, 0 wallclock secs ( 0.03 usr 0.00 sys + 0.21 cusr 0.14 csys = 0.38 CPU)
Result: FAIL

AIX 5.3.0.0/TL12-05 IBM,9115-505 PowerPC_POWER5/1898(2) 3920 Mb

$ ./perl -I../lib -V
Summary of my perl5 (revision 5 version 21 subversion 5) configuration:
Snapshot of: f63a114b48be4e9c44534b6e07ecdeea8be2ef71
Platform:
osname=aix, osvers=5.3.0.0, archname=aix-thread-multi-64all
uname='aix i3 3 5 0004898ad300 '
config_args='-des -Dcc=gcc -Dusedevel -Duseithreads -Duse64bitall'
hint=recommended, useposix=true, d_sigaction=define
useithreads=define, usemultiplicity=define
use64bitint=define, use64bitall=define, uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler:
cc='gcc -maix64', ccflags ='-D_THREAD_SAFE -maix64 -DPERL_DONT_CREATE_GVSV -D_ALL_SOURCE -D_ANSI_C_SOURCE -D_POSIX_SOURCE -DUSE_NATIVE_DLOPEN -DNEED_PTHREAD_INIT -fno-strict-aliasing -pipe -I/pro/local/include -maix64 -DUSE_64_BIT_ALL -D_FORTIFY_SOURCE=2',
optimize='-O',
cppflags='-D_THREAD_SAFE -maix64 -DPERL_DONT_CREATE_GVSV -D_ALL_SOURCE -D_ANSI_C_SOURCE -D_POSIX_SOURCE -DUSE_NATIVE_DLOPEN -DNEED_PTHREAD_INIT -fno-strict-aliasing -pipe -I/pro/local/include'
ccversion='', gccversion='4.2.4', gccosandvers=''
intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=87654321
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=8, longdblkind=-1
ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
alignbytes=8, prototype=define
Linker and Libraries:
ld='gcc -maix64', ldflags =' -L/usr/local/ppc64/lib64 -Wl,-b64 -L/pro/local/lib -Wl,-brtl -Wl,-bdynamic -Wl,-b64'
libpth=/usr/local/ppc64/lib /usr/lib /usr/local/ppc64/lib64 /lib /usr/ccs/lib /usr/local/lib /usr/lib64
libs=-lbind -lnsl -ldbm -ldb -ldl -lld -lm -lcrypt -lpthreads -lc
perllibs=-lbind -lnsl -ldl -lld -lm -lcrypt -lpthreads -lc
libc=/lib/libc.a, so=a, useshrplib=false, libperl=libperl.a
gnulibc_version=''
Dynamic Linking:
dlsrc=dl_aix.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Xlinker -bE:/opt/perl64/lib/5.21.5/aix-thread-multi-64all/CORE/perl.exp'
cccdlflags=' ', lddlflags=' -Wl,-b64 -Wl,-bhalt:4 -Wl,-G -Wl,-bI:$(PERL_INC)/perl.exp -Wl,-bE:$(BASEEXT).exp -Wl,-bnoentry -lpthreads -lc -lm -L/usr/local/ppc64/lib64 -L/pro/local/lib'


Characteristics of this binary (from libperl):
Compile-time options: HAS_TIMES MULTIPLICITY PERLIO_LAYERS
PERL_DONT_CREATE_GVSV
PERL_HASH_FUNC_ONE_AT_A_TIME_HARD
PERL_IMPLICIT_CONTEXT PERL_MALLOC_WRAP
PERL_NEW_COPY_ON_WRITE PERL_PRESERVE_IVUV
PERL_USE_DEVEL USE_64_BIT_ALL USE_64_BIT_INT
USE_ITHREADS USE_LARGE_FILES USE_LOCALE
USE_LOCALE_COLLATE USE_LOCALE_CTYPE
USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_PERLIO
USE_PERL_ATOF USE_REENTRANT_API
Locally applied patches:
SMOKEf63a114b48be4e9c44534b6e07ecdeea8be2ef71
Built under aix
Compiled at Oct 3 2014 13:36:07
@INC:
../lib
/opt/perl64/lib/site_perl/5.21.5/aix-thread-multi-64all
/opt/perl64/lib/site_perl/5.21.5
/opt/perl64/lib/5.21.5/aix-thread-multi-64all
/opt/perl64/lib/5.21.5
.
--
H.Merijn Brand http://tux.nl Perl Monger http://amsterdam.pm.org/
using perl5.00307 .. 5.19 porting perl5 on HP-UX, AIX, and openSUSE
http://mirrors.develooper.com/hpux/ http://www.test-smoke.org/
http://qa.perl.org http://www.goldmark.org/jeff/stupid-disclaimers/

Rafael Garcia-Suarez

unread,
Oct 3, 2014, 8:07:07 AM10/3/14
to H.Merijn Brand, perl5-...@perl.org
Ah, of course. I now have made the tests a bit more tolerant
with df968918245d10232f955ab0965da7f8d6297a29.

Leon Timmermans

unread,
Oct 3, 2014, 8:30:01 AM10/3/14
to H.Merijn Brand, Perl5 Porters
Of course. POSIX doesn't specify these text, but they tend to be highly conserved among Unices. Except AIX has to be a  unique snowflake whenever the spec allows for it…

Leon

Ricardo Signes

unread,
Oct 3, 2014, 9:06:49 AM10/3/14
to Rafael Garcia-Suarez, H.Merijn Brand, perl5-...@perl.org
* Rafael Garcia-Suarez <r...@consttype.org> [2014-10-03T08:07:07]
> Ah, of course. I now have made the tests a bit more tolerant
> with df968918245d10232f955ab0965da7f8d6297a29.

Your fix seems A-OK, but in case you did want to verify the message more, I
found it useful in the past (91514b78, e.g.) to get the errstr explicitly, then
test for that.

--
rjbs
signature.asc

Michael Felt

unread,
Oct 3, 2014, 3:00:07 PM10/3/14
to Ricardo Signes, Rafael Garcia-Suarez, H.Merijn Brand, perl5-...@perl.org
This is from a smoke in progress (out of the log file)
1st one of:
root@x113:[/data/smoker/run]./smokestatus.pl -r -m
./smokestatus.pl-0.014 Test::Smoke-1.61_03 Test::Smoke::Reporter-0.053

Checking status for configuration 'AIX_config' (blead)
  Change number 4cd408bae5cd62850a437b986c09084b3eea6338 started on Fri Oct  3 08:00:11 2014.
    9 out of 12 configurations finished in 9 hours 58 minutes.
    9 configurations showed failures (F).
    1 failure in the running configuration.
    3 configurations to finish, estimated completion in 2 hours 29 minutes
    Average smoke duration: 1 hour 6 minutes.
  Matrix, using cc -q64 version 11.1.0.19:
  v5.21.4-221-g4cd408b  Configuration (common) none
  ----------- ---------------------------------------------------------
  F F F F     -Duse64bitall -Duselongdouble
  F F F F     -Duse64bitall -Duse64bitint
  F F F F     -Duse64bitall
  F F F F     -Duseithreads -Duselongdouble
  F ? F F     -Duseithreads -Duse64bitint
  | | | +----- PERLIO = perlio -DDEBUGGING
  | | +------- PERLIO = stdio  -DDEBUGGING
  | +--------- PERLIO = perlio
  +----------- PERLIO = stdio
on AIX: 6100-06-08-1216

p.s. - lots of recurring errors because I do not have C++ installed, I am just ignoring them personally.

TSTENV = stdio  The _test target is deprecated. Please upgrade your smoker

# Failed test 30 - <<>> does not treat - as STDIN at io/argv.t line 191
#      got "Can\'t open -: A file or directory in the path name does not exist. at -e line 1.\n"
# expected "Can\'t open -: No such file or directory at -e line 1.\n"
# Failed test 31 - <<>> does not treat - as STDIN at io/argv.t line 199
#      got "Can\'t open : A file or directory in the path name does not exist. at -e line 1.\n"
# expected "Can\'t open : No such file or directory at -e line 1.\n"
t/io/argv ..................................................... FAILED at test 30
# using temp dir /data/smoker/perl-current/cpan/Module-Metadata/MMD-eNn73mX5
# removing temp dir /data/smoker/perl-current/cpan/Module-Metadata/MMD-eNn73mX5
Hexadecimal number > 0xffffffff non-portable at t/refaddr.t line 24.
Hexadecimal number > 0xffffffff non-portable at t/refaddr.t line 24.
Hexadecimal number > 0xffffffff non-portable at t/refaddr.t line 24.
Hexadecimal number > 0xffffffff non-portable at t/refaddr.t line 24.
Hexadecimal number > 0xffffffff non-portable at t/refaddr.t line 24.
cc: 1501-228 (W) input file c++ not found
Compilation cannot proceed because one or more IBM compiler product files are missing or corrupted.  R
einstall the complete IBM XL C/C++ for AIX V11.1 compiler.
cc: 1501-228 (W) input file c++ not found
Compilation cannot proceed because one or more IBM compiler product files are missing or corrupted.  R
einstall the complete IBM XL C/C++ for AIX V11.1 compiler.
# Will use Digest::MD5
Failed 1 test out of 2181, 99.95% okay.
....

Error while reading test-results: 2
Extending failures with harness:
        ../t/io/argv.t
# Failed test 30 - <<>> does not treat - as STDIN at ../t/io/argv.t line 191

#      got "Can\'t open -: A file or directory in the path name does not exist. at -e line 1.\n"
# expected "Can\'t open -: No such file or directory at -e line 1.\n"
# Failed test 31 - <<>> does not treat - as STDIN at ../t/io/argv.t line 199

#      got "Can\'t open : A file or directory in the path name does not exist. at -e line 1.\n"
# expected "Can\'t open : No such file or directory at -e line 1.\n"
# Failed test 32 - <<>> does not treat - as STDIN at ../t/io/argv.t line 205

#      got "Can\'t open : A file or directory in the path name does not exist. at -e line 1.\n"
# expected "Can\'t open : No such file or directory at -e line 1.\n"
# Failed test 33 - <<>> does not treat ...| as fork at ../t/io/argv.t line 216

#      got "Can\'t open echo foo |: A file or directory in the path name does not exist. at -e line 1.
\n"
# expected "Can\'t open echo foo |: No such file or directory at -e line 1.\n"
# Failed test 34 - <<>> does not treat ...| as fork after eof at ../t/io/argv.t line 223

#      got "Can\'t open echo foo |: A file or directory in the path name does not exist. at -e line 1,
 <> line 3.\n"
# expected "Can\'t open echo foo |: No such file or directory at -e line 1, <> line 3.\n"
../t/io/argv.t ..

Failed 5/36 subtests

Test Summary Report
-------------------
../t/io/argv.t (Wstat: 0 Tests: 36 Failed: 5)
  Failed tests:  30-34
Files=1, Tests=36,  0 wallclock secs ( 0.03 usr  0.01 sys +  0.21 cusr  0.12 csys =  0.37 CPU)
Result: FAIL

    ../t/io/argv.t..............................................FAILED
        30-34

This repeats several times, only real change is the name of /data/smoker/perl-current/cpan/Module-Metadata/MMD-eNn73mX5

And, it has been occurring in all AIX versions - 5.3 TL7, AIX 5.3 TL12, AIX 6.1 TL{6,7,8}, AIX 7.1 TL{1,2} - in short all of my smokers.

(FYI - I am getting some battery errors, so I am turning my Power5 systems off to get them out of my "noise container" and change the batteries. For a few days there will be reports only for 5.3 TL12, 6.1 TL8, and 7.1 TL3. And, when I am working and travelling these only run as cronjobs - I am not trying to understand why they failed - but I had noticed this had been going on for a few days, and not in the Linux ones - as frequently)







0 new messages