Reading files

[Josef Eschgfaeller]

Is it not possible to read a file at once instead of using read-line
or read-char? The following is very slow for big files.
I'm working with GCL under Linux. Thanks.

(defmacro readfile (name)
`(with-open-file (file ,name :direction :input)
(do ((row (read-line file) (read-line file nil 'file-end))
(text "" (format nil "~a~a~%" text row)))
((eql row 'file-end) text))))

J. Eschgfaeller

[Erik Naggum]

* Josef Eschgfaeller <e...@felix.unife.it>

| Is it not possible to read a file at once instead of using read-line
| or read-char?

two ways: (1) use READ-SEQUENCE on a pre-allocated string (use
FILE-LENGTH to learn how big it should be), or (2) make a FFI call to
the mmap system call and treat it as a string. the first is simple.

| I'm working with GCL under Linux. Thanks.

there are much better Common Lisp implementations available under Linux.
I would recommend Allegro CL from Franz Inc. visit www.franz.com.

#:Erik
--
@1999-07-22T00:37:33Z -- pi billion seconds since the turn of the century

[Kent M Pitman]

Josef Eschgfaeller <e...@felix.unife.it> writes:

> Is it not possible to read a file at once instead of using read-line

> or read-char? The following is very slow for big files.

The code you uses uses an O(n^2) string copying, so it's not surprising
that it's slow.

> I'm working with GCL under Linux. Thanks.
>

> (defmacro readfile (name)
> `(with-open-file (file ,name :direction :input)
> (do ((row (read-line file) (read-line file nil 'file-end))
> (text "" (format nil "~a~a~%" text row)))
> ((eql row 'file-end) text))))
>
> J. Eschgfaeller

(There's also zero justification for this being a macro. It will only
bloat your code and the savings in speed is negligible. A function call
is probably set up in only a couple of instructions, and anything
that is doing this many calls to read-line, open, format, etc. is using
so many other instructions that the value of having this be a macro
rapidly approaches zero.)

Even without doing anything else, you could switch it to
an O(n) algorithm (where n is the number of lines) by doing:

(defun read-file (filename)
(with-open-file (stream filename :direction :input)
(read-stream stream)))

(defun read-stream (stream)
(with-output-to-string (result-stream)
(loop (multiple-value-bind (line eof-p)
(read-line stream nil nil)
(when line (write-string line result-stream))
(when eof-p (return))
(terpri result-stream)))))

However, if you need greater efficiency, you should consider using
READ-SEQUENCE instead of READ-LINE, so that you can buffer larger
blocks of text than single lines.

See CLHS for info on READ-SEQUENCE.
http://www.harlequin.com/education/books/HyperSpec/FrontMatter/index.html