Clearing eof condition from stream.
Kaz Kylheku
has indicated EOF, when more data becomes available. In that case you
want the language-level stream to be able to forget that the EOF happened
and try the input operation again.
I can't find anything in the HyperSpec about how an end of input
indication affects the state of a Lisp stream. Is there an end-of-file
state that can be removed, so that further input operations on the file
or device can be attempted?
Erik Naggum
| I can't find anything in the HyperSpec about how an end of input
| indication affects the state of a Lisp stream. Is there an end-of-file
| state that can be removed, so that further input operations on the file
| or device can be attempted?
My understanding is that such "error states" are not remembered, because
you are supposed to actually deal with an error condition, which is quite
unlike the situation in a language where errors are routinely ignored and
it would be a service to the programmer to let him clear the error/eof
flag once some part of the program that cared about such things had time
to look at it. So look at it as if the handler in both languages clears
the error flag, but you always handle an error immediately in Common Lisp
and so decoupling the error flag from the handler makes no sense.
///
--
Norway is now run by a priest from the fundamentalist Christian People's
Party, the fifth largest party representing one eighth of the electorate.
--
Carrying a Swiss Army pocket knife in Oslo, Norway, is a criminal offense.
Barry Margolin
>* Kaz Kylheku
>| I can't find anything in the HyperSpec about how an end of input
>| indication affects the state of a Lisp stream. Is there an end-of-file
>| state that can be removed, so that further input operations on the file
>| or device can be attempted?
>
> My understanding is that such "error states" are not remembered, because
> you are supposed to actually deal with an error condition, which is quite
> unlike the situation in a language where errors are routinely ignored and
> it would be a service to the programmer to let him clear the error/eof
> flag once some part of the program that cared about such things had time
That's how it works in C, you have to call clearerr(). It's not done
automatically for you, AFAIK.
> to look at it. So look at it as if the handler in both languages clears
> the error flag, but you always handle an error immediately in Common Lisp
> and so decoupling the error flag from the handler makes no sense.
How does the EOF error handler clear the EOF condition in CL once it has
handled it? E.g. if you expect that someone has since appended data to the
file you were reading, and you want to read what they've added (like the
"tail -f" command on Unix).
--
Barry Margolin, bar...@genuity.net
Genuity, Woburn, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.
Kaz Kylheku
> So look at it as if the handler in both languages clears
> the error flag, but you always handle an error immediately in Common Lisp
> and so decoupling the error flag from the handler makes no sense.
Understood; thanks for that information. So I think it might be a bug
in CLISP's implementation that if you evaluate:
(progn (read t nil) (read t nil) (read t nil))
and I indicate EOF on the terminal, all three reads bail immediately,
and the block returns NIL.
The end-of-file condition is handled internally in the first one,
but then it occurs again again in the second one and in the third.
But I know that this isn't how terminal devices work on the system I'm
using; an EOF just turns into a read() system call that returns zero
bytes read. Nothing is recorded by the operating system; you can keep
reading from the tty. So the behavior is being provided somewhere
in the language implementation.
According to your description, behavior like this is explained by
the terminal device behaving as if EOF is final; the abstract semantics
of the stream keep no such sticky information.
Interstingly, you can abuse the clear-input function into influencing
the behavior:
(progn (read t nil) (clear-input)
(read t nil) (clear-input)
(read t nil))
now you type Ctrl-D three times in a row to allow this form to return.
The HyperSpec's description of clear-input doesn't mention any effect
upon any end-of-file state, so this entirely a CLISP-specific hack.
Erik Naggum
| How does the EOF error handler clear the EOF condition in CL once it has
| handled it?
I wonder how I managed _not_ to explain this clearly. There is no "EOF
condition" on the stream to _clear_ in the first place. When the handler
returns or transfers control somewhere else, the condition is _gone_. If
the stream remembers that it has hit end of file and you cannot retry the
operation, I consider that a violation of conformance with the standard.
Kaz Kylheku
>> to look at it. So look at it as if the handler in both languages clears
>> the error flag, but you always handle an error immediately in Common Lisp
>> and so decoupling the error flag from the handler makes no sense.
>
>How does the EOF error handler clear the EOF condition in CL once it has
>handled it?
From Eric's description, it's clear that this is implicit. There is
simply no such indication. The stream comes across an error, and
generates a condition.
This is analogous to a POSIX file descriptor returning zero bytes
read; it doesn't remember that it did that. It's an event, rather
than state.
>E.g. if you expect that someone has since appended data to the
>file you were reading, and you want to read what they've added (like the
>"tail -f" command on Unix).
If I understand it, CL implementation of tail -f wouldn't have to do
anything special other than catch the end-of-file condition. E.g.
(loop
(handler-case (read-more-and-print stream)
(end-of-file () (sleep 1))))
Kent M Pitman
There are no streams defined by CL that do this. It would be an
implementation-defined behavior and you'd want to use
implementation-defined operators. Ask your vendor.
It wouldn't surprise me if some vendors just say "go ahead and try the
operation again, no action is needed". Such a treatment would seem
conforming, though not uniquely so. So even if that felt to you
as "portable" because no implementation-defined operators were used, it
still would not be portable because the language is silent on whether
that's enough.
That's my personal opinion anyway. For whatever that's worth.
Kent M Pitman
>
> In article <32139902...@naggum.net>, Erik Naggum wrote:
> > So look at it as if the handler in both languages clears
> > the error flag, but you always handle an error immediately in Common Lisp
> > and so decoupling the error flag from the handler makes no sense.
>
> Understood; thanks for that information. So I think it might be a bug
> in CLISP's implementation that if you evaluate:
>
> (progn (read t nil) (read t nil) (read t nil))
>
> and I indicate EOF on the terminal, all three reads bail immediately,
> and the block returns NIL.
I guess it depends on the implementation, but IMO eof on the terminal is
like typing control-d at the shell. It means end-of-session, NOT no
more input available. When you're looking to find more input that might
arrive later, you should (loop until (listen)) if you have no multiprocessing
or use some appropriate process-wait kind of thing if you do have
multiprocessing.
I certainly expect that the above will bail immediately when an EOF occurs.
> The end-of-file condition is handled internally in the first one,
> but then it occurs again again in the second one and in the third.
As I think it should.
> But I know that this isn't how terminal devices work on the system I'm
> using; an EOF just turns into a read() system call that returns zero
> bytes read. Nothing is recorded by the operating system; you can keep
> reading from the tty. So the behavior is being provided somewhere
> in the language implementation.
You aren't supposed to be worrying about what happens in system call land.
Thatw ill only confuse you. What happens at the operating system level
is utterly irrelevant.
EOF means the stream is done. Resuming a done stream is a recovery action,
not a normal mode of operation. The only place I can think of where I'd
maybe expect this to get used is keep-alive support on http connections or,
if you have a pop-up window that is a resourced thing you aren't going to
recreate every time, you want to clear the eof before re-using the box.
But ordinarily if you're just pausing for kbd input, that is NOT an eof.
That's just data not yet ready and LISTEN is the answer.
> According to your description, behavior like this is explained by
> the terminal device behaving as if EOF is final; the abstract semantics
> of the stream keep no such sticky information.
If you keep no sticky info, then I recommend you view the stream as either
always open (and so no more input means it's just not yet ready, as in a
normal keybaord stream).
> Interstingly, you can abuse the clear-input function into influencing
> the behavior:
>
> (progn (read t nil) (clear-input)
> (read t nil) (clear-input)
> (read t nil))
This ought make no difference and IMO should be a bug.
> now you type Ctrl-D three times in a row to allow this form to return.
> The HyperSpec's description of clear-input doesn't mention any effect
> upon any end-of-file state, so this entirely a CLISP-specific hack.
Not hack. Bug. If you're at EOF, there is no input pending. All
clear-input does is clear input. An eof is not input and out not be
clearable. That's my interpretation anyway.
There is no agency which is the final authority on opinions, so the
CLISP people could assert my opinion of what ANSI CL intends is wrong
and there would be no way to say who was right. Put another way: It's
an opinion whose reading is right, but within my reading, at least,
this is not open to opinion. :-)
Wade Humeniuk
I actually had to code this case recently. I ended up writing this function
as read-line would properly get the input and then signal an eof condition
on the next read-line. *standard-input* was bound to the console input in
Windows. It seems read-line would block on the next try after the eof
condition.
LWW 4.1.20
(defun read-from-outside-world ()
(loop for line = (read-line *standard-input* nil)
do
(cond
((null line) nil)
((char= #\# (char line 0)) nil)
(t (return-from read-from-outside-world (read-from-string (format
nil "(~A)" line)))))))
Wade
Espen Vestre
> How does the EOF error handler clear the EOF condition in CL once it has
> handled it? E.g. if you expect that someone has since appended data to the
> file you were reading, and you want to read what they've added (like the
> "tail -f" command on Unix).
This naive implementation of 'tail -f' works fine as far as I can tell:
(defun tail-f (in out)
(with-open-file (f in)
(loop
(handler-case
(format out "~&~a~%" (read-line f))
(end-of-file (sleep 1))))))
--
(espen)
Barry Margolin
> If
> the stream remembers that it has hit end of file and you cannot retry the
> operation, I consider that a violation of conformance with the standard.
I just took a quick look in CLTL2 and CLHS and couldn't find where it says
that you're allowed to keep reading after END-OF-FILE is signalled. Since
there is no CLEAR-EOF or CLEAR-ERROR, does this implicitly mean that these
are unnecessary and it happens automatically? I don't think this is
obvious, and it would be better to state it explicitly.
Sam Steingold
> * On the subject of "Re: Clearing eof condition from stream."
> * Sent on Mon, 05 Nov 2001 23:36:00 GMT
> * Honorable k...@ashi.footprints.net (Kaz Kylheku) writes:
>
> So I think it might be a bug
> in CLISP's implementation that if you evaluate:
I told you already (in <clisp-list>) that this has been fixed in the
CLISP sources on 2001-10-15.
--
Sam Steingold (http://www.podval.org/~sds)
Keep Jerusalem united! <http://www.onejerusalem.org/Petition.asp>
Read, think and remember! <http://www.iris.org.il> <http://www.memri.org/>
God had a deadline, so He wrote it all in Lisp.
Kaz Kylheku
>> * In message <AnFF7.18988$Ud.6...@news1.rdc1.bc.home.com>
>> * On the subject of "Re: Clearing eof condition from stream."
>> * Sent on Mon, 05 Nov 2001 23:36:00 GMT
>> * Honorable k...@ashi.footprints.net (Kaz Kylheku) writes:
>>
>> So I think it might be a bug
>> in CLISP's implementation that if you evaluate:
>
>I told you already (in <clisp-list>) that this has been fixed in the
>CLISP sources on 2001-10-15.
This is a separate issue, at least from my perspective: what I wrote
to the list about was propagation of EOF to the interactive loop.
At that time I wrote that I had no idea whether or not an end-of-file
state lingers in a Common Lisp stream, which is what I was asking
about here.
Kent M Pitman
> In article <32139931...@naggum.net>, Erik Naggum <er...@naggum.net> wrote:
> > If
> > the stream remembers that it has hit end of file and you cannot retry the
> > operation, I consider that a violation of conformance with the standard.
>
> I just took a quick look in CLTL2 and CLHS and couldn't find where it says
> that you're allowed to keep reading after END-OF-FILE is signalled. Since
> there is no CLEAR-EOF or CLEAR-ERROR, does this implicitly mean that these
> are unnecessary and it happens automatically? I don't think this is
> obvious, and it would be better to state it explicitly.
It's extremely rare that Barry says something that I so much disagree
with. His point of view is usually different than mine, and quite
interestingly so, but ordinarily I can understand where he's coming
from and can endorse his views, even where they conflict with mine.
On this point, though, I have a real philosophical problem with the
above statement. So pardon me, Barry, for drawing swords on this one.
It's nothing personal. ;-)
Pragmatically, even, I guess I'd agree if I'd read some spec somewhere
for the things you're talking about and knew how other systems work.
But a priori in an abstract way, you are presuming that this is an
event and that it must be resolved.
I regard eof as a position in a file. The move me forward operation moves
you. The "am I at the wall?" operation does not. In my mind, there is
nothing vague about saying that if you try to read a char and are at end
of stream, you get back NIL. It doesn't say it moves you to some other
place than end of file.
Consider a list implemented as a stack. No doc anywhere says that if
you do (setq x (list 'a)) (pop x) => A (pop x) => NIL that you must
now clear the stack-empty situation. The fact is that NIL is NIL and
it will keep being NIL as many times as you check.
You could CREATE a stack based on some physics model in which the
stack was actively harmed by an attempt to overpop (the C language
would simulate this by going ahead and moving the machine pointer into
the stack to some otherwise-protected passowrd area for the
convenience of people trying to break into the system, for example).
But I don't think it's fair to assume that just because some low-level
OS designer on one particular OS (even if it is probably one of teh
two OS's that seems to breed people who thinkt here was never any
other OS *than* that OS) or some language designer in one popular
language implemented it that way, that it must be assumed to be
defaultly that way in all OS's or all languages.
Back to streams, the active form of this is that if you try to read
and you set eof-error-p to true, you'll get an error, but the error is
not about the fact of the eof. The eof is just sitting there in front
of you. The error is about the fact that you tried to read into it.
When you're not trying to read into it, you're back where you started,
looking at the eof.
I can see how people with other event models could confuse you on this point
and how you might wish there to be tutorial information, but I just don't
see any problem at all in the spec here because the spec does not create
the concepts you're using to confuse yourself and so the spec has no
responsibility to resolve your confusion over those concepts.
Just my opinion, of course. (In other words, I like to think I would
take this position if I had not written the spec and that I am not
merely being defensive about my personal wording choice, which on
other occasions I'm quick to identify as crufty if I think I've made a
mistake or not had enough time to do a reasonable job. There is, of course,
no way to totally clear me of guilt on any potential defensiveness charge.)
The fairest reason you could give for why there should be explicit verbiage
on this point is the empirical observation that use of the languags and
operating systems where this occurs has bred in a kind of tunnel vision
in which it's just too hard for users of that system to adapt to other
ways of thinking without additional help, help that those same languages
and operating systems would never think twice about denying the Lisp
community, who are similarly at a disadvantage in the other direction.
Fortunately, in the other direction, the folks used to those other systems
are often more adaptable and get by without wording fixes to confusions
of their own devisings, because there are certainly plenty of those going
unanswered.
I'm sorry for sounding a little harsh on this, but CL is a language that is
unlike most other languages in that it was designed to be IMPLEMENTATION
INDEPENDENT. And cries for Lisp to retroactively be chided for not figuring
out what language would be dominant today and cater to it specially, or cries
for Lisp to acknowledge that one language or system has won for all time
even though history says that things change and so might change again,
really bug me. Or maybe I just didn't get enough sleep. You tell me.