Re: [Pugs] Closing a file handle surprised the heck out of me
Autrijus Tang
> But it gets worse.
>
> my $lines = [ =$fh ];
> seek($fh, 0);
> my $lines2 = [ =$fh ];
> close $fh;
>
> $lines2 must somehow remember that seek has happened.
That is fine because the three thunks are registered to the fh
in evaluation order. What will be more fun is if they are all
part of some other lazy lists, which may be accessed in some
unpredictable order.
That is why lazy languages typically use some sort of typechecking to
avoid mixing computations with actions... :)
Thanks,
/Autrijus/
Autrijus Tang
> On Mon, Mar 14, 2005 at 10:58:00PM +1100, Andrew Savige wrote:
> : my $fh = open(@ARGS[0]);
> : my @lines = =$fh;
> : $fh.close();
> : for @lines { print"$_" }
> :
>
> Hmm. It's probably a design bug. I'm wondering if we have to specify
> that ordinary assignment not only copies logically but, in fact, always
> does ** flattening (immediate flattening) like Perl 5
Except ** does not flatten recursively, so:
my $lines = [ =$fh ];
is still borked.
Do we want a *** "nuke everything flat" semantic device, if not the
operator? Or an eager context on the right hand side of "="?
On the other hand, we can alternatively specify that closing a
file handle must force all thunks associated to it, i.e. in this
case fill @lines with real data. Pugs can do that just fine, but I
have no idea how parrot is to achieve that...
Thanks,
/Autrijus/
Luke Palmer
Perhaps lists by assignment are COW lazy. So they behave just like
ordinary COW objects, except they can be lazy at the end:
my @a = gather {
for 1... {
say;
take;
}
};
@a[10]; # says 1 2 3 ... 10
@a[5] = 42;
@a[20]; # nothing printed
I wonder if that's DingTRT though...
Luke
>
> Thanks,
> /Autrijus/
Larry Wall
: On Mon, Mar 14, 2005 at 08:06:08AM -0800, Larry Wall wrote:
: > On Mon, Mar 14, 2005 at 10:58:00PM +1100, Andrew Savige wrote:
: > : my $fh = open(@ARGS[0]);
: > : my @lines = =$fh;
: > : $fh.close();
: > : for @lines { print"$_" }
: > :
: >
: > Hmm. It's probably a design bug. I'm wondering if we have to specify
: > that ordinary assignment not only copies logically but, in fact, always
: > does ** flattening (immediate flattening) like Perl 5
:
: Except ** does not flatten recursively, so:
:
: my $lines = [ =$fh ];
:
: is still borked.
:
: Do we want a *** "nuke everything flat" semantic device, if not the
: operator? Or an eager context on the right hand side of "="?
I don't like ***. As with :::: in rules, at some point you bail out
and go alphabetic. So you might write that as
my $lines = [ $fh.slurp ]
That doesn't work so well for
my $lines = [ <>.slurp ]
though...
: On the other hand, we can alternatively specify that closing a
: file handle must force all thunks associated to it, i.e. in this
: case fill @lines with real data. Pugs can do that just fine, but I
: have no idea how parrot is to achieve that...
I think we have to recognize that close() is a very non-FP-ish
operation. It's essentially a kind of database commit, so that you
or someone else can reliable open the file and know what's going to
be there. (We've essentally got the same problem writing lazy lists
to a file.) So I guess the question is whether we can intuit the
intent of the programmer with the close. There's the "I'm tired of
this filehande so close it logically but don't flush anything" vs the
"close this Right Now and flush everything Right Now and maybe even
fsync to disk to make jolly sure we're in a known state."
Since the earlier sentiment can be handled merely by letting the handle
go out of scope and get garbage collected someday, we probably have
to treat close as the more violent intention. So if we can autoflush
output, we can also autofill input, but probably only for iterators
that were used in list context. An iterator in scalar context expects
the close to truncate the rest of the input. A Perl 5 programmer
expects that
$foo = <$input>;
will stop working if you close $input, but will expect
@foo = <$input>;
to produce all the values as if it had been eagerly flattened, because
that's what list context means in Perl 5. In Perl 6, list context
merely means the promise of flattening, but in the case of
$foo = [ =$fh ];
it means we have to fulfill the promise when $fh is closed. It really is
a kind of COW situation, where close is construed as a write to the
effective length of the input. The situation doesn't apply to
$foo = [ 1... ];
because by default there's no handle to close on the infinite sequence.
Just as well, I suppose...
If we can capture the COW semantics, then we wouldn't necessarily have
to fix assignment, but it might be a good idea in any case to define
@foo = (bar())
as
@foo := [**bar()]
just to force programmers to make their eager/lazy intent clearer by
distinguishing = from :=. Then the fact that = also does a top-level
copy just sort of naturally falls out.
Larry
Larry Wall
: Perhaps lists by assignment are COW lazy. So they behave just like
: ordinary COW objects, except they can be lazy at the end:
:
: my @a = gather {
: for 1... {
: say;
: take;
: }
: };
: @a[10]; # says 1 2 3 ... 10
: @a[5] = 42;
: @a[20]; # nothing printed
:
: I wonder if that's DingTRT though...
My feeling is that it prints
11 12 13 14 15 16 17 18 19 20
and then if you say ~@a[0..19] you'll get:
1 2 3 4 5 42 7 8 9 10 11 12 13 14 15 16 17 18 19 20
That is, once a value is generated, it's just an ordinary array value.
The generator is just attached at the end, and doesn't care what you
do with existing values.
On the other hand, maybe you have to use "my @a :=" if we make = eager.
Larry
Larry Wall
tends to argue for eager =. Plus there's the fact that it's the sort
of restriction we can relax if we figure out how, but it would be hard
to impose if people started to rely on lazy assignment and then we
decided it was a bad idea.
Larry
Nigel Hamilton
> :
> : my $fh = open(@ARGS[0]);
> : my @lines = =$fh;
> : $fh.close();
> : for @lines { print"$_" }
> :
> :
> : pugs t.p6 t.p6
> :
> : produces no output. Move $fh.close() to after the for
> : loop and all is well. Is this a bug?
>
I wonder if IO::All could provide some inspiration here? Not so much for
solving this specific bug - but making easy IO things easy and making file
handle funkiness hard?
Just a meme for the pool ...
Nige
Larry Wall
: Yep. Does that put =, := etc into a category of operators that
: are hard to (usefully) overload in classes? I mean the eager =
: doesn't look like letting the left and right hand side have much
: to do or decide in the process---other than provide and swallow
: the value or complain about not beeing able to due so.
I'd hazard a guess that it constrains the allowable signatures for
such an operator.
Larry
Luke Palmer
I use = as my no-thinking, catch-all assignment. I'd really like lazy
lists to be pervasive enough that I don't have to think about them any
differently than I think about any other kind of list.
Making = lazy and making = eager both have the same disadvantage, but in
different places. If we make = lazy, then people who don't think about
lazy lists will use it in modules and whatnot, and it will (at best) die
when they try to write them to files or print them or do other things
that infinite lists can't really do. On the other hand, if we make =
eager, then people who write modules who don't think about lazy lists
will be denying the users of their module lazy access when it would be
perfectly acceptable.
And I can't decide which is worse. But I think the latter is worse.
The former puts the decision of how to use lazy lists on the module
user, where the latter puts the decision on the module author. Just
like C<use fatal>, we want the user to call the shots, because they're
the one who knows how the module fits with the rest of the program.
I think that as long as lazy lists die when you do things that you can't
do to them to them (sic.), and they die with a nice error message that
says where they were created, that lazy assignment should be fine. We
who write perl6 will have to work a little harder, but we know that from
Perl's philosophy anyway. And by harder, I mean to make the "close"
example work; to make a filehandle understand that it has lazy lists
attached to it and to flush itself when necessary.
Of course, there are dragons hiding in that too. If you tie a lazy list
to a filehandle and only ever plan to use its first two elements, and
you close the filehandle while the list is still in scope, then a 100
megabyte file will give you trouble you never asked for.
Maybe it's best to let the array decide. There are some lazy lists
which are healthy to throw around recklessly, like C<1...>. There are
others, like C<< <> >>, which have nasty ordering dependencies, and are
best kept at the expression level and not assigned (only bound... but
your ordering dependencies can bite you there, too).
If arrays are lazy by default, that basically mean they're behaving as
tied values (because you can do anything while you generate), and that
we might as well outright remove the C<is tied> optimization
restriction. And then we should remove it for scalars and hashes too,
and (please) let it propogate through assignment if it wants to. I've
always wondered how much we could actually optimize if we know things
aren't tied anyway.... Junctions already do tie-like things without
being tied (even though it's the *value* which is doing those things).
Whew... stop for a breath here... </ramble>
This is a hard problem. And it touches on a lot of my active concerns.
For the time being, I'll buy your argument that it's easy to relax
assigment to lazy, but hard to harden it up to eager.
Luke
Thomas Sandlaß
Yep. Does that put =, := etc into a category of operators that
are hard to (usefully) overload in classes? I mean the eager =
doesn't look like letting the left and right hand side have much
to do or decide in the process---other than provide and swallow
the value or complain about not beeing able to due so.
Regards,
--
TSa (Thomas Sandlaß)
Miroslav Silovic
>On the other hand, we can alternatively specify that closing a
>file handle must force all thunks associated to it, i.e. in this
>case fill @lines with real data. Pugs can do that just fine, but I
>have no idea how parrot is to achieve that...
>
>
But it gets worse.
my $lines = [ =$fh ];
seek($fh, 0);
my $lines2 = [ =$fh ];
close $fh;
$lines2 must somehow remember that seek has happened.
Miro
Larry Wall
: On Mon, Mar 14, 2005 at 05:28:29PM +0100, Miroslav Silovic wrote:
: > But it gets worse.
: >
: > my $lines = [ =$fh ];
: > seek($fh, 0);
: > my $lines2 = [ =$fh ];
: > close $fh;
: >
: > $lines2 must somehow remember that seek has happened.
:
: That is fine because the three thunks are registered to the fh
: in evaluation order. What will be more fun is if they are all
: part of some other lazy lists, which may be accessed in some
: unpredictable order.
You could treat seek as a synchronization point like close.
:
: That is why lazy languages typically use some sort of typechecking to
: avoid mixing computations with actions... :)
Which is also what we're doing, except that we're hiding that fact
from the user whenever we can rather than rubbing their nose in it. :-)
Larry
Autrijus Tang
> : That is fine because the three thunks are registered to the fh
> : in evaluation order. What will be more fun is if they are all
> : part of some other lazy lists, which may be accessed in some
> : unpredictable order.
>
> You could treat seek as a synchronization point like close.
True. So maybe the trick is define such a set of synchronization
points for resources that has lazy/eager conflicts, instead of forcing
eager context on RHS of all infix:<=>, especially because most
assignments probably only involve simple COW copying, so it'd be
sad to lose the laziness benefit.
> : That is why lazy languages typically use some sort of typechecking to
> : avoid mixing computations with actions... :)
>
> Which is also what we're doing, except that we're hiding that fact
> from the user whenever we can rather than rubbing their nose in it. :-)
As long as surprises can be minimized (or at least explained), that's
entirely fine, I think. :)
Thanks,
/Autrijus/