Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

reading file to list

1,677 views
Skip to first unread message

Drew Krause

unread,
Jan 16, 2009, 3:29:37 PM1/16/09
to
OK, I want to create a nested list in Lisp (always of only integers) from a
text file, such that each line in the text file would be represented as a
sublist in the 'imported' list.

Like this:

(defun make-list-from-text filename)
....

"blob.txt" is
3 10 2
4 1
11 18

so..
(make-list-from-text "blob.txt") => ((3 10 2) (4 1) (11 18))

Any help would be greatly appreciated!

Thanks, DK

w_a_...@yahoo.com

unread,
Jan 16, 2009, 4:16:13 PM1/16/09
to

Ruby:

IO.readlines("blob.txt").map{|line| line.split }

blandest

unread,
Jan 16, 2009, 5:05:24 PM1/16/09
to

(defun make-list-from-file (filename)
(with-open-file (in filename)
(loop for line = (read-line in nil nil)
while line
collect (read-from-string (concatenate 'string "(" line
")")))))
CL-USER> (make-list-from-file "tmp")

Thomas A. Russ

unread,
Jan 16, 2009, 5:17:17 PM1/16/09
to
Drew Krause <drkr...@mindspring.com> writes:

What have you got so far?

--
Thomas A. Russ, USC/Information Sciences Institute


Ross Lonstein

unread,
Jan 16, 2009, 8:08:02 PM1/16/09
to
Here's my take on it....

(defun split (delim line)
"Return a list of subsequences separated by a one character delimiter, delim itself is not returned"
(loop :for mark = 0 :then (1+ point)
:for point = (position delim line :start mark)
:collect (subseq line mark point)
:while point))

(defun make-list-from-text (fn)
"assemble list of lists of numbers parsed from file, posed by drkrause on cll 2009-01-16"
(with-open-file (stream fn)
(loop :for line = (read-line stream nil nil)
:while line
:collect (mapcar #'parse-integer (split #\Space line)))))

CL-USER> (make-list-from-file "/tmp/blob.txt")


((3 10 2) (4 1) (11 18))

- Ross

jos...@corporate-world.lisp.de

unread,
Jan 16, 2009, 10:36:22 PM1/16/09
to
On Jan 16, 11:17 pm, t...@sevak.isi.edu (Thomas A. Russ) wrote:

Let's try a more exotic 'solution':

On Jan 16, 10:16 pm, w_a_x_...@yahoo.com wrote:


> On Jan 16, 2:29 pm, Drew Krause <drkra...@mindspring.com> wrote:
>
>
>
> > OK, I want to create a nested list in Lisp (always of only integers) from a
> > text file, such that each line in the text file would be represented as a
> > sublist in the 'imported' list.
>
> > Like this:
>
> > (defun make-list-from-text filename)
> > ....
>
> > "blob.txt" is
> > 3 10 2
> > 4 1
> > 11 18
>
> > so..
> > (make-list-from-text "blob.txt") => ((3 10 2) (4 1) (11 18))
>
> > Any help would be greatly appreciated!
>

> > Thanks, DK
>
> Ruby:
>
> IO.readlines("blob.txt").map{|line| line.split }

Why not try something new? No need to allocate lines, split the and
then parse integer from the parts.

(defun read-lines (stream &aux result list (rt (copy-readtable)))
(flet ((newline-reader (stream char)
(declare (ignore stream char))
(when list
(push (nreverse list) result)
(setf list nil))
nil))
(set-macro-character #\newline #'newline-reader nil rt)
(let ((*readtable* rt))
(loop for item = (read stream nil stream)
until (eq item stream)
when item do (push item list)
finally (when list (push (nreverse list) result)))
(nreverse result))))


William James

unread,
Jan 17, 2009, 12:45:39 AM1/17/09
to
w_a_...@yahoo.com wrote:

If you want to convert the strings to integers:

IO.readlines("blob.txt").map{|line| line.split.map{|s| s.to_i }}

budden

unread,
Jan 17, 2009, 7:02:32 AM1/17/09
to
Hi!
There are multiple split-sequence library implementations, no need to
write one's own. E.g.

http://www.cliki.net/SPLIT-SEQUENCE

I wonder why everyone suggest one's own split-sequence function
instead of pointing OP to the existing library. There are more
interesting tasks for lisp coding that reinventing a wheel.

Ross Lonstein

unread,
Jan 17, 2009, 9:41:48 AM1/17/09
to
budden <budde...@mail.ru> writes:
[snip]

> I wonder why everyone suggest one's own split-sequence function
> instead of pointing OP to the existing library.

My thinking was that if the OP can't manage writing this particular bit
of code, they probably haven't worked out ASDF yet. I expected that
someone would point it out.

> There are more interesting tasks for lisp coding that reinventing a
> wheel.

Compared to SPLIT-SEQUENCE, a caster.

And answering on c.l.l. is already a form of recreation (pun intended)
;)

- Ross

Xah Lee

unread,
Jan 17, 2009, 12:16:07 PM1/17/09
to
comp.lang.lisp,comp.lang.scheme,comp.lang.functional,comp.lang.python,comp.lang.ruby

Here's a interesting toy problem posted by Drew Krause to
comp.lang.lisp:

------------------------
On Jan 16, 2:29 pm, Drew Krause wrote [paraphrased a bit]:

OK, I want to create a nested list in Lisp (always of only integers)
from a text file, such that each line in the text file would be
represented as a sublist in the 'imported' list.

example of a file's content

3 10 2
4 1
11 18

example of programing behavior


(make-list-from-text "blob.txt") => ((3 10 2) (4 1) (11 18))

-----------------
Here's a emacs lisp version:

(defun read-lines (file)
"Return a list of lines in FILE."
(with-temp-buffer
(insert-file-contents file)
(split-string
(buffer-substring-no-properties 1 (point-max)) "\n" t)
)
)

(defvar mylist '() "result list" )
(setq mylist '()) ; init in case eval'd again

(mapc
(lambda (x) (setq mylist
(cons (split-string x " ") mylist )) )
(read-lines "xxblob.txt")
)

The above coding style is a typical maintainable elisp.

In a show-off context, it can be reduced to by about 50%, but still
far verbose than ruby or say perl (which is 1 or 2 lines. (python
would be 3 or 5)).

Note that the result element is string, not numbers. There's no easy
way to convert them on the fly. 3 or so more lines will be needed to
do that.

The “read-lines” function really should be a built-in part of elisp.
It is not so mostly because emacs people have drawn themselves into a
elitist corner, closing off to the outside world. (i am sending a
suggestion to the official emacs dev list on adding read-line now.)

-----------------

w_a_x_...@yahoo.com gave a ruby solution:

IO.readlines("blob.txt").map{|line| line.split }

augumented by Jilliam James for result to be numbers:

IO.readlines("blob.txt").map{|line| line.split.map{|s| s.to_i }}

Very beautiful.

-------------------

That's really the beauty of Ruby.

This problem and ruby code illustrates 2 fundamental problems of lisp,
namely, the cons problem, and the nested syntax pain. Both of which
are practically unfixable.

The lisp's cons fundamentally makes nested list a pain to work with.
Lisp's nested syntax makes functional sequencing cumbersome.

In the ruby code, its post-fix sequential notation (as a side effect
of its OOP notation) brings out the beauty of functional sequencing
paradigm (sometimes known as functional chain, sequencing, filtering,
unix piping).

its list, like all modern high level langs such as perl, php, python,
javascript, don't have the lisp's cons problem. The cons destroys the
usability of lists up-front, untill you have some at least 2 full-time
years of coding lisp to utilize cons properly. (and even after that,
it is still a pain to work with, and all you gain is a bit of speed
optimization in rare cases that requires largish data, most of which
has better solutions such as a database.)

Both of these problems i've published articles on.

For more detail on the cons problem, see
the section “The Cons Business” at

• Fundamental Problems of Lisp
http://xahlee.org/UnixResource_dir/writ/lisp_problems.html

For more detail on the nested syntax problem for function chaining,
see
the section “How Purely Nested Notation Limits The Language's Utility”
at:

• The Concepts and Confusions of Prefix, Infix, Postfix and Fully
Nested Notations
http://xahlee.org/UnixResource_dir/writ/notations.html

Xah
http://xahlee.org/


Xah Lee

unread,
Jan 17, 2009, 12:34:39 PM1/17/09
to
On Jan 17, 9:16 am, Xah Lee <xah...@gmail.com> wrote:
> Here's a interesting toy problem posted by Drew Krause to
> comp.lang.lisp:
> ...

The code in my previous elisp code got a bump. It should be:

(defun read-lines (file)
"Return a list of lines in FILE."
(with-temp-buffer
(insert-file-contents file)
(split-string
(buffer-substring-no-properties 1 (point-max)) "\n" t)
)
)

(mapcar
(lambda (x) (split-string x " ") )
(read-lines "xxblob.txt")
)

The article is now archived at:

• A Ruby Illustration of Lisp Problems
http://xahlee.org/UnixResource_dir/writ/lisp_problems_by_ruby.html

Xah
http://xahlee.org/


Xah Lee

unread,
Jan 17, 2009, 12:55:43 PM1/17/09
to
On Jan 17, 9:34 am, Xah Lee <xah...@gmail.com> wrote:
> The code in my previous elisp code got a bump. It should be:
> ...

> • A Ruby Illustration of Lisp Problems
> http://xahlee.org/UnixResource_dir/writ/lisp_problems_by_ruby.html

Sorry again. More correction:

(defun read-lines (file)
"Return a list of lines in FILE."
(with-temp-buffer
(insert-file-contents file)
(split-string
(buffer-substring-no-properties 1 (point-max)) "\n" t)
)
)

(mapcar
(lambda (x)
(mapcar
(lambda (y) (string-to-number y) )
(split-string x " ")


)
)
(read-lines "xxblob.txt")
)

this is last post of correction.

Xah
http://xahlee.org/


André Thieme

unread,
Jan 17, 2009, 3:30:49 PM1/17/09
to
Xah Lee schrieb:

> comp.lang.lisp,comp.lang.scheme,comp.lang.functional,comp.lang.python,comp.lang.ruby
>
> Here's a interesting toy problem posted by Drew Krause to
> comp.lang.lisp:
>
> ------------------------
> On Jan 16, 2:29 pm, Drew Krause wrote [paraphrased a bit]:
>
> OK, I want to create a nested list in Lisp (always of only integers)
> from a text file, such that each line in the text file would be
> represented as a sublist in the 'imported' list.
>
> example of a file's content
>
> 3 10 2
> 4 1
> 11 18
>
> example of programing behavior
> (make-list-from-text "blob.txt") => ((3 10 2) (4 1) (11 18))
>
> w_a_x_...@yahoo.com gave a ruby solution:
>
> IO.readlines("blob.txt").map{|line| line.split }
>
> augumented by Jilliam James for result to be numbers:
>
> IO.readlines("blob.txt").map{|line| line.split.map{|s| s.to_i }}
>
> Very beautiful.
>
> -------------------
>
> That's really the beauty of Ruby.
>
> This problem and ruby code illustrates 2 fundamental problems of lisp,
> namely, the cons problem, and the nested syntax pain. Both of which
> are practically unfixable.

*Of course* Xah is wrong, as always.
Who would expect anything else?
In the Lisp style Clojure for example one does exactly the same as
Jillian James (JJ) did in Ruby:
(map #(map (fn [s] (Integer/parseInt s)) (.split % "\\s")) (line-seq
(reader "blob.txt")))

This is slightly longer, simply because the functions have longer names.
Integer/parseInt vs to_i

Also the function split takes a regexp, so I have to add the "\\s" here.
I don’t know though if Jillians version also handles any kind of
whitespac per line.
And the last thing is, that the function reader returns a BufferedReader.
So in general this is more valuable in real programs as opposed to one-
line scripts. If we combine line-seq and reader into readline and apply
the two other changes we would say:
(map #(map (fn [s] (to_i s)) (.split %)) (readlines "blob.txt"))
vs


IO.readlines("blob.txt").map{|line| line.split.map{|s| s.to_i }}

So, obviously Xah is far away from any reality.
It took him hours of his life to write up all his ideas about Lisp.
Sad to see that all that time was wasted. All of it was wrong.
Poor Xah :(


And btw, Drew Krause is just a Lisp newbie. No Lisper would ever store
data for his programs in the format


3 10 2
4 1
11 18

but instead as


(3 10 2)
(4 1)
(11 18)

And then read it back into the program with:
(map read-string (line-seq (reader "blob.txt")))


André
--

K Livingston

unread,
Jan 17, 2009, 7:25:50 PM1/17/09
to
On Jan 16, 9:36 pm, "jos...@corporate-world.lisp.de" <jos...@corporate-

world.lisp.de> wrote:
> Why not try something new? No need to allocate lines, split the and
> then parse integer from the parts.
>
> (defun read-lines (stream &aux result list (rt (copy-readtable)))
>   (flet ((newline-reader (stream char)
>            (declare (ignore stream char))
>            (when list
>              (push (nreverse list) result)
>              (setf list nil))
>            nil))
>     (set-macro-character #\newline #'newline-reader nil rt)
>     (let ((*readtable* rt))
>       (loop for item = (read stream nil stream)
>             until (eq item stream)
>             when item do (push item list)
>             finally (when list (push (nreverse list) result)))
>       (nreverse result))))


I agree, no need to read and re-read everything, like all the split
people want. The lisp reader can be made to take care of this so
easily. But why does everyone want to traverse the data at least 2
times. (in the above case 3, but the last push/nreverse could be
trivially changed to a collect to avoid that)

CommonLisp give us READ-DELIMITED-LIST - we don't need to define it,
or bring in a package to do it. Lisp readers are highly optimized,
let's let it do the work for us.

Now unfortunately, read-delimited-list reads right though whitespace
so we can't trivially say
(read-delimited-list #\newline stream)
as it won't see the newline like we want it to.

But that's easy enough to fix, let's just change the readtable
temporarily to think that newline is a terminating, non-constituent
character. What else is one of those? right-parenthesis will do, so
let's just make it behave like that temporarily. And since we don't
want to do this over and over and over again, let just store it in a
variable (it could be in a closure around the function too if you
wanted to be fancier, and if you didn't ever need this readtable for
anything else, but... that's for another day).

(defvar *newline-not-white-readtable*
(let ((*readtable* (copy-readtable nil)))
(set-syntax-from-char #\newline #\))
*readtable*))

Ok now we have a readtable, lets make a function to read a single
line...

Inside the function, we'll temporarily switch to our new readtable, so
that we can stop when we see a newline. Also, since read-delimited-
list doesn't fail gracefully, we will have to catch the EOF condition
it generates. (There might be an easier/shorter way to do that, but
this works, and just returns the stream when EOF is encountered. Why
can't it return nil? OP didn't say what should happen with an empty
line, e.g. just a newline character, so assuming that is valid, and
since we have to look for something anyway, it may as well be the
stream.)

(defun read-list-insides (s)
(let ((*readtable* *newline-not-white-readtable*))
(handler-bind
((end-of-file #'(lambda (c)
(declare (ignore c))
(return-from read-list-insides s))))
(read-delimited-list #\newline s))))


Now to get all the lines in one list.

(with-open-file (s *data-file*)
(loop for item = (read-list-insides s)
until (eq item s)
collect item))

There it is - one pass, and barely any code.


Questions for the reader/OP:

1. if you don't really want to collect all the values/lines but just
operate on them one at a time, how could you change the loop call, or
what else could you do.

2. if you do need to collect them all, is it worth moving the loop
inside of the let for the readtable and the handler-bind. Does that
buy you anything?

3. what happens if there isn't an newline at the end of the file / at
the end of the last sequence of numbers? does this code expect there
to be one?


good luck,
Kevin

Xah Lee

unread,
Jan 17, 2009, 7:30:24 PM1/17/09
to
Xah Lee wrote:
> • A Ruby Illustration of Lisp Problems
> http://xahlee.org/UnixResource_dir/writ/lisp_problems_by_ruby.html


On Jan 17, 12:30 pm, André Thieme <address.good.until.
2009.may...@justmail.de> wrote:


> In the Lisp style Clojure for example one does exactly the same as
> Jillian James (JJ) did in Ruby:

> (map #(map (fn [s] (Integer/parseInt s)) (.split % "\\s")) (line-seq
> (reader "blob.txt")))

Note that you have nested map. This is a problem of function chaning
as i detailed.

The other problem of cons, is commonly seen. Just about every week, we
see some perhaps beginning lisper asking how to do various trivial
list
manipulation problem, which you don't see such question or frequency
in any of modern high level lang forms.

The frequently asked list manipulation question we see include how to
append, prepend, or anything involving nested list such as
partitioning, reordering sublists, getting leaves, interpretation of
leaves, etc. This is caused by the cons.

associated lisp problem compound the issue. Namely, the not exactly
regular syntax, and the eval model of sometimes symbol sometimes
uneval'd symbol, e.g. “'(...)”, “`(...,@ ...)” things.

The clojure example you gave above, apparently inherited the irregular
syntax problem. (you see the #, [], % things, which is breaks the
lisp's sexp idea) Also, all modern lisp basically all get fucked up by
inheriting the cons (e.g. clojure, NewLisp, Arc lisp, Liskell). Often,
they change the semantic a bit apparently as a mesaure towards solving
the cons problem. (and Qi Lisp creates a huge bag of ad hoc, extremely
ugly, syntax soup)

Advice: if you are creating a lispy lang, two things:

• stick to a _pure_ nested syntax, no exception whatsoever. (e.g. no
more ` ' # % shits) If you want, add a transparent layer on top to
support arbitrary algeraic notation. (e.g. look at Mathematica, CGOL)

• get rid of the cons. (you can still implement the so-called linked
list, but no where it should be shown to the programer)

Further readings:

Xah
http://xahlee.org/

K Livingston

unread,
Jan 17, 2009, 7:45:53 PM1/17/09
to
On Jan 17, 2:30 pm, André Thieme <address.good.until.

2009.may...@justmail.de> wrote:
> And btw, Drew Krause is just a Lisp newbie. No Lisper would ever store
> data for his programs in the format
> 3 10 2
> 4 1
> 11 18
>
> but instead as
> (3 10 2)
> (4 1)
> (11 18)

Maybe he doesn't get to designate the format he has to read?

But really, why wouldn't a Lisper consider that format? If it really
is just sequences of numbers you're just adding a lot of funny
parenthesis in your data file, making it larger for no good reason,
when you already have delimiters.

Assuming he does get to pick the format and it is just numbers, why
not just store them as sequences of bytes/integers etc. instead of
text?

All that said, with usual combined goals of laziness (for the effort
of the programmer) and the desire to have something (mostly) human
readable/editable, I usually go with the format you are advocating.

But, I think people are selling lisp short / forgetting the tools they
have, when they assume or promote that there needs to be parenthesis
in a data file for lisp code to read it easily or efficiently.

at least, that's my two cents,
Kevin

André Thieme

unread,
Jan 17, 2009, 8:07:58 PM1/17/09
to
K Livingston schrieb:

> On Jan 17, 2:30 pm, André Thieme <address.good.until.
> 2009.may...@justmail.de> wrote:
>> And btw, Drew Krause is just a Lisp newbie. No Lisper would ever store
>> data for his programs in the format
>> 3 10 2
>> 4 1
>> 11 18
>>
>> but instead as
>> (3 10 2)
>> (4 1)
>> (11 18)
>
> Maybe he doesn't get to designate the format he has to read?
>
> But really, why wouldn't a Lisper consider that format? If it really
> is just sequences of numbers you're just adding a lot of funny
> parenthesis in your data file, making it larger for no good reason,
> when you already have delimiters.

If it were really like that, if those parenthesis would be there for
no good reason, then it would make no sense, I agree.
But they are there for a reason.
Their presence makes it easier to read them back.
No splitting needed, no conversion to integers, not two mappings.
And writing out the data is also trivial.
For small amounts of data that’s all we need. If we want bigger
datasets there is always a DB system waiting.


André
--

jos...@corporate-world.lisp.de

unread,
Jan 17, 2009, 8:14:23 PM1/17/09
to
On Jan 18, 1:45 am, K Livingston <kevinlivingston.pub...@gmail.com>
wrote:

If I would make it easy for Lisp, then I would make it just one list:

((3 10 2)(4 1)(11 18))

or

((3 10 2)
(4 1)
(11 18))

or

(


(3 10 2)
(4 1)
(11 18)

)

Then I would use just one READ.


André Thieme

unread,
Jan 17, 2009, 8:25:00 PM1/17/09
to
Xah Lee schrieb:

> Xah Lee wrote:
>> • A Ruby Illustration of Lisp Problems
>> http://xahlee.org/UnixResource_dir/writ/lisp_problems_by_ruby.html
>
>
> On Jan 17, 12:30 pm, André Thieme <address.good.until.
> 2009.may...@justmail.de> wrote:
>
>
>> In the Lisp style Clojure for example one does exactly the same as
>> Jillian James (JJ) did in Ruby:
>
>> (map #(map (fn [s] (Integer/parseInt s)) (.split % "\\s")) (line-seq
>> (reader "blob.txt")))
>
> Note that you have nested map. This is a problem of function chaning
> as i detailed.

Yes, Jillian also has nested maps:


IO.readlines("blob.txt").map{|line| line.split.map{|s| s.to_i }}

> The other problem of cons, is commonly seen. Just about every week, we
> see some perhaps beginning lisper asking how to do various trivial
> list
> manipulation problem, which you don't see such question or frequency
> in any of modern high level lang forms.

You see questions about trivial problems every week in all forums about
programming languages.
It has nothing to do with conses. Wrong road.


> The frequently asked list manipulation question we see include how to
> append, prepend, or anything involving nested list such as
> partitioning, reordering sublists, getting leaves, interpretation of
> leaves, etc. This is caused by the cons.

Yes, same with all containers in all programming languages.


> The clojure example you gave above, apparently inherited the irregular
> syntax problem. (you see the #, [], % things, which is breaks the
> lisp's sexp idea)

My code used 8 “mysterious symbols”:
( ) # [ ] . " %

The Ruby version had these 7:
( ) | { } . "

And of course, neither in Ruby nor Clojure they break any idea.

> Also, all modern lisp basically all get fucked up by
> inheriting the cons (e.g. clojure, NewLisp, Arc lisp, Liskell). Often,
> they change the semantic a bit apparently as a mesaure towards solving
> the cons problem. (and Qi Lisp creates a huge bag of ad hoc, extremely
> ugly, syntax soup)

Funny.
What you write is an english text with words that most people can
understand.
You trick them into thinking that it actually makes sense what you say.
It is so technical, that a random would believe it.
But what you really say is about as meaningful as saying that it is
impossible to write good programs in Python, because it adds whitespace
to its syntax.

You wasted lots of your time. Or was this maybe generated by a bot?
Maybe http://pdos.csail.mit.edu/scigen/ or something like that?
I also found this paper that you wrote:
http://apps.pdos.lcs.mit.edu/scicache/184/scimakelatex.7076.Xah+Lee.html
Another crap posting of you. But a random person might be impressed.


André
--

Xah Lee

unread,
Jan 17, 2009, 10:07:24 PM1/17/09
to

a idiot wrote:
> Yes, Jillian also has nested maps:

the issue here, is not about whether Ruby has nested map or not. It is
about illustrating a lisp problem. In particular, nested syntax
impedes the functional programing paradigm of function chaining.

a idiot wrote:
> My code used 8 “mysterious symbols”:
> ( ) # [ ] . " %
>
> The Ruby version had these 7:
> ( ) | { } . "

The issue here is not about which lang employs more special chars. The
issue is about the irregularities in lisp's syntax damaged its power
of its otherwise believed superior regular syntax.

Xah
http://xahlee.org/

K Livingston

unread,
Jan 17, 2009, 10:10:23 PM1/17/09
to
On Jan 17, 7:07 pm, André Thieme <address.good.until.

2009.may...@justmail.de> wrote:
> K Livingston schrieb:
>
>
>
> > On Jan 17, 2:30 pm, André Thieme <address.good.until.
> > 2009.may...@justmail.de> wrote:
> >> And btw, Drew Krause is just a Lisp newbie. No Lisper would ever store
> >> data for his programs in the format
> >> 3 10 2
> >> 4 1
> >> 11 18
>
> >> but instead as
> >> (3 10 2)
> >> (4 1)
> >> (11 18)
>
> > Maybe he doesn't get to designate the format he has to read?
>
> > But really, why wouldn't a Lisper consider that format?  If it really
> > is just sequences of numbers you're just adding a lot of funny
> > parenthesis in your data file, making it larger for no good reason,
> > when you already have delimiters.
>
> If it were really like that, if those parenthesis would be there for
> no good reason, then it would make no sense, I agree.
> But they are there for a reason.
> Their presence makes it easier to read them back.
> No splitting needed, no conversion to integers, not two mappings.


The read-delimited-list solution above works in a single pass and is
not hindered in the slightest by the absence of parentheses. (and
it's only a few lines of code)


> And writing out the data is also trivial.

for the data format above mapping
(format stream "~{~S~^ ~}~%" list-of-numbers)
seems pretty trivial.

or if you really want it in one line
(format stream "~:{~@{~S~^ ~}~%~}" list-of-list-of-numbers)


Anyway, all I've been trying to point out is that the absence or
presence of parentheses has pretty much no bearing on ones ability to
read, manipulate, or print data. (The belief that it does comes up
often.) For the most part any straightforward data format is as
trivial as any other to do whatever you want with. The lisp reader by
default reads s-expressions, but that doesn't mean that's all you can
input in lisp. A few tweaks here or there to the reader covers most
of the cases I've ever seen. CommonLisp gives you a lot of power by
exposing all these tools to you, and I, for one, would rather go back
to my toolbox and pick up my screwdriver, than sit there hammer in
hand and say "why are all these screws in my way."

-Kevin

Xah Lee

unread,
Jan 18, 2009, 3:31:15 AM1/18/09
to
On Jan 17, 10:25 am, Tino Wildenhain <t...@wildenhain.de> wrote:
> > [[int(x) for x in line.split()] for line in open("blob.txt")]

Nice (python code).

Few comments:

• the above code is borderline of atypical. e.g. it is not a average
python code would produce or one'd seen in corporate python code.

• voodoo like the above makes me dislike python. To me, the one
advantage of python is its clarity enforced by its syntax.
Specifically, the forced indendation and quite simple semantics.
However, the way i've seen Guido's propensities and how python 3 is
moving to, it is becoming more mumbo jumbo of computer sciency OOP
jargons with syntax soup. (with iterators, enumerators, list
comprehension... shits forced upon the users)

The above line illustrate well the ad hoc syntax soup nature python is
moving into.

Further readings:

• Perl-Python Tutorial: List Comprehension
http://xahlee.org/perl-python/list_comprehension.html

• Lambda in Python 3000
http://xahlee.org/perl-python/python_3000.html

Xah
http://xahlee.org/


Rainer Joswig

unread,
Jan 18, 2009, 4:39:39 AM1/18/09
to
In article
<3e5f02cb-2a50-4013...@q30g2000prq.googlegroups.com>,
K Livingston <kevinliving...@gmail.com> wrote:

> On Jan 16, 9:36 pm, "jos...@corporate-world.lisp.de" <jos...@corporate-
> world.lisp.de> wrote:
> > Why not try something new? No need to allocate lines, split the and
> > then parse integer from the parts.
> >
> > (defun read-lines (stream &aux result list (rt (copy-readtable)))
> >   (flet ((newline-reader (stream char)
> >            (declare (ignore stream char))
> >            (when list
> >              (push (nreverse list) result)
> >              (setf list nil))
> >            nil))
> >     (set-macro-character #\newline #'newline-reader nil rt)
> >     (let ((*readtable* rt))
> >       (loop for item = (read stream nil stream)
> >             until (eq item stream)
> >             when item do (push item list)
> >             finally (when list (push (nreverse list) result)))
> >       (nreverse result))))

...

> CommonLisp give us READ-DELIMITED-LIST - we don't need to define it,
> or bring in a package to do it. Lisp readers are highly optimized,
> let's let it do the work for us.
>

...

>
> 3. what happens if there isn't an newline at the end of the file / at
> the end of the last sequence of numbers? does this code expect there
> to be one?


I thought about the read-delimited-list version, too.
But it does not return a list when it sees an
eof during read. So a line with numbers with
no newline at the end can't be read with it.

It might (?) be useful to have another version
of read-delimited list, where the end of the list could
be determined by other factors: amount of
items read, sequence as end marker, eof, eol, predicate
on reading an item ... It might even have a filter than does
include only those elements that pass a filter.

Left as an programming exercise. ;-)

--
http://lispm.dyndns.org/

William James

unread,
Jan 18, 2009, 5:53:59 PM1/18/09
to
André Thieme wrote:

> K Livingston schrieb:
> > On Jan 17, 2:30 pm, André Thieme <address.good.until.
> >2009.may...@justmail.de> wrote:
> > > And btw, Drew Krause is just a Lisp newbie. No Lisper would ever
> > > store data for his programs in the format
> > > 3 10 2
> > > 4 1
> > > 11 18
> > >
> > > but instead as
> > > (3 10 2)
> > > (4 1)
> > > (11 18)
> >
> > Maybe he doesn't get to designate the format he has to read?
> >
> > But really, why wouldn't a Lisper consider that format? If it
> > really is just sequences of numbers you're just adding a lot of
> > funny parenthesis in your data file, making it larger for no good
> > reason, when you already have delimiters.
>
> If it were really like that, if those parenthesis would be there for
> no good reason, then it would make no sense, I agree.
> But they are there for a reason.
> Their presence makes it easier to read them back.
> No splitting needed, no conversion to integers, not two mappings.
> And writing out the data is also trivial.

In other words, don't try to do anything in Lisp that isn't trivial.
The data has to be changed to suit Lisp; the Lisp program can't be
changed to suit the data.

> For small amounts of data that’s all we need. If we want bigger
> datasets there is always a DB system waiting.

For non-trivial tasks, don't attempt to use Lisp. It's too CLumsy.
Use a database.

William James

unread,
Jan 18, 2009, 6:29:57 PM1/18/09
to
André Thieme wrote:

> Xah Lee schrieb:
> > comp.lang.lisp,comp.lang.scheme,comp.lang.functional,comp.lang.pytho

That fails when numbers are separated by more than one space.
And what about leading or trailing space?

>
> This is slightly longer, simply because the functions have longer
> names. Integer/parseInt vs to_i
>
> Also the function split takes a regexp, so I have to add the "\\s"
> here. I don’t know though if Jillians version also handles any
> kind of whitespac per line.


irb(main):003:0> puts " foo \t bar "
foo bar
=> nil
irb(main):004:0> " foo \t bar ".split
=> ["foo", "bar"]
irb(main):005:0> "foo...bar?-!hoo".split /\W+/
=> ["foo", "bar", "hoo"]


> And the last thing is, that the function reader returns a
> BufferedReader. So in general this is more valuable in real programs
> as opposed to one- line scripts. If we combine line-seq and reader
> into readline and apply the two other changes we would say:
> (map #(map (fn [s] (to_i s)) (.split %)) (readlines "blob.txt"))

That is not a complete program.

> vs
> IO.readlines("blob.txt").map{|line| line.split.map{|s| s.to_i }}

That is the complete program.

>
> So, obviously Xah is far away from any reality.
> It took him hours of his life to write up all his ideas about Lisp.
> Sad to see that all that time was wasted. All of it was wrong.
> Poor Xah :(
>
>
> And btw, Drew Krause is just a Lisp newbie. No Lisper would ever store
> data for his programs in the format
> 3 10 2
> 4 1
> 11 18
>
> but instead as
> (3 10 2)
> (4 1)
> (11 18)

Perhaps the data was stored by someone else. Understand?

>
> And then read it back into the program with:
> (map read-string (line-seq (reader "blob.txt")))
>

You make a very strong case that Lisp is very feeble at
processing data. I'm almost convinced.

Ruby isn't feeble, so data like this is fine:


shall we begin?
or lotus135? 1984 times!
The 3 stooges: COBOL,LISP,FORTRAN.
3.14, si11y L00KING


Extracting the unsigned integers:

IO.readlines('data').map{|s| s.scan(/\d+/).map{|s| s.to_i}}
==>[[], [135, 1984], [3], [3, 14, 11, 0]]

André Thieme

unread,
Jan 18, 2009, 9:24:22 PM1/18/09
to
William James schrieb:

> André Thieme wrote:
>
> You make a very strong case that Lisp is very feeble at
> processing data. I'm almost convinced.

I somehow don’t believe you :-)


> Ruby isn't feeble, so data like this is fine:
>
>
> shall we begin?
> or lotus135? 1984 times!
> The 3 stooges: COBOL,LISP,FORTRAN.
> 3.14, si11y L00KING
>
>
> Extracting the unsigned integers:
>
> IO.readlines('data').map{|s| s.scan(/\d+/).map{|s| s.to_i}}
> ==>[[], [135, 1984], [3], [3, 14, 11, 0]]

Just take the program I posted before and replace
(.split % "\\s")
with
(re-seq #"\d+" %)


Now that I was so kind to give you what you asked for, you will
probably do the same for me and show me your Ruby code, in which you
define a variable that contains the Fibonacci sequence.

(def fibs (lazy-cat [0 1] (map + fibs (drop 1 fibs))))


(nth fibs 70) ==> 190392490709135

One requirement is that when you look at the n'th fib the results up
to n must be memoized, so that nothing needs to be calculated again
when you ask for a fib between 0 and n. And when you ask for the n+1'th
it will start the calculation right from point n.

Example:
user> (time (nth fibs 900))
"Elapsed time: 16.898726 msecs"

user> (time (nth fibs 901))
"Elapsed time: 0.268603 msecs"


André
--

Daniel Weinreb

unread,
Jan 19, 2009, 6:10:03 PM1/19/09
to
Drew Krause wrote:
> OK, I want to create a nested list in Lisp (always of only integers) from a
> text file, such that each line in the text file would be represented as a
> sublist in the 'imported' list.
>
> Like this:
>
> (defun make-list-from-text filename)
> ....
>
> "blob.txt" is
> 3 10 2
> 4 1
> 11 18
>
> so..

> (make-list-from-text "blob.txt") => ((3 10 2) (4 1) (11 18))
>
> Any help would be greatly appreciated!
>
> Thanks, DK

(Sorry, I posted this with the wrong subject before.)

blandest: That's nice and short. In the Olde Days, we would have
criticized the storage allocation done by the concatentation, but
these days it's not a big deal. I wish Common Lisp had a more
succinct way to do (concatenate 'string ...). You neglected to turn
the strings into integers but that's trivial.

Russ Lonstein: Excellent, provided you use the split-sequence library
as discussed.

Ranier Joswig: Using a reader macro that does a side-effect at
newline is novel; I would not have thought of that.

Xah Lee: Elisp is an interesting choice. But without converting the
strings to integers, it's not really right.

It's interesting that Elisp already has a style of reading in the
whole file, but no built-in to convert strings to integers, whereas
Common Lisp is the other way around.

The statement "It is not so mostly because emacs people have drawn
themselves into a elitist corner, closing off to the outside world" is
just a rude ad hominem attack.

Your claim that this shows something wrong with lists is completely
unclear, although if I read your paper (I'll try) I might have some
idea what you're getting at

Andre Thieme: There's no reason to be insulting. First of all, who
says the file was stored by a Lisp program? Lisp should be able to
read and parse anything. Second, if you are interested in Lisp, we
should be welcoming new users, not insulting them. The best way for
Lisp to become more acceptable is for there to be more users.

Ken Livingston: I think use of the readtable is more straightforward
than what Ranier did. And remembering that read-delimited-list exists
is great; I think we all forgot about that.

Stylistically, it's a bit strange to use the stream as a special
returned value. I do understand that you're using the stream as an
object that can't possibly be the outcome of read-delimited-list, but
I personally am not a fan of special return values. Also, the
exception handling can use handler-case instead of handler-bind, which
is simpler and less code. Sadly, Common Lisp does not define the
condition that is signaled upon end of file. And the filename ought
to be an argument, not a global. Maybe something like this?

(defun read-lines (filename)
(with-open-file (stream filename)
(loop for item = (handler-case
(read-delimited-list #\newline stream)
(t () (return-from read-lines result)))
collect item into result)))

Now, suppose we wanted to do it the way the Ruby code does it. Assume
we have a built-in function called read-lines that takes a filename,
opens the file, reads all the way to the end, and returns one string
for each line.

(mapcar (lambda (line)
(mapcar #'parse-integer (split-sequence:split-sequence line)))
(read-lines "blob.txt"))

Ruby has shorter names than Lisp. If we compensate for that, just to
narrow down where the differences are:

(mapcar (lambda (line)
(mapcar #'to-i (split line)))
(readlines "blob.txt"))

We can make it look more like Ruby by reversing the order of arguments
to mapcar and writing it all on one line. First add function "mp",
which is map with its arguments reversed.

(defun mp (list function) (mapcar function list))

Now, here's how the Common Lisp looks next to the Ruby:

(mp (readlines "blob.txt") (lambda (line) (mp (split line) #'to-i)))


IO.readlines("blob.txt").map{|line| line.split.map{|s| s.to_i }}

These are pretty similar. So Lisp is missing read-lines, has longer
names, and takes the arguments to the mapping function backwards from
that of Ruby. Also, Ruby's "to_i" is probably a generic, less elegant
to implement in Common Lisp because not everything is an instance of a
class.

The reason Lisp mapping functions take the function first is so that
they can provide many lists, if the function takes more than one
argument. I don't know how this is handled in Ruby.

Note that these issues are specific to Common Lisp. Another dialect
of Lisp could have the above changes and still be faithful to Lisp.

Xah Lee

unread,
Jan 19, 2009, 7:59:41 PM1/19/09
to
Hi Daniel Weinreb,

Xah wrote:
> • A Ruby Illustration of Lisp Problems
> http://xahlee.org/UnixResource_dir/writ/lisp_problems_by_ruby.html

Daniel Weinreb wrote:
> Xah Lee: Elisp is an interesting choice. But without converting the
> strings to integers, it's not really right.

i did post a correction quickly that includes the conversion to string/
num. Here's the full code:

(defun read-lines (file)
"Return a list of lines in FILE."
(with-temp-buffer
(insert-file-contents file)
(split-string

(buffer-string) "\n" t)
)
)

(mapcar
(lambda (x)
(mapcar
(lambda (y) (string-to-number y) )
(split-string x " ")
)
)

(read-lines "blob.txt")
)

> It's interesting that Elisp already has a style of reading in the
> whole file, but no built-in to convert strings to integers, whereas
> Common Lisp is the other way around.
>
> The statement "It is not so mostly because emacs people have drawn
> themselves into a elitist corner, closing off to the outside world" is
> just a rude ad hominem attack.

sure. In a political context, many criticism or description of the
situation from one party can be seen as ad hominem attack. I feel that
that many old emacs users, which includes significant portion of emacs
developers (if not majority), are so exteremly bottled up and turn
down any possibility of advancing. They are killing emacs. (similar
feelings for most regular posters here in comp.lang.lisp )

Now, that statement is imprecise, is attacking, needs a lot
qualifications. I have literally written about 20 articles each a
thousand words or more over the past 3 years that explains many
specific cases in painful detail. (e.g. see http://xahlee.org/emacs/emacs_essays_index.html
) However, in a one sentence description in this thread's context,
calling them “elitist” and “closing off to the outside world”, is to
the point.

> Your claim that this shows something wrong with lists is completely
> unclear, although if I read your paper (I'll try) I might have some
> idea what you're getting at

More specifically, 2 fundamental problems of lisp i feel this ruby
example illustrates well:

• the cons impedes many aspects of lists. e.g. difficult to learn,
confusing, hard to use, prevent development of coherent list
manipulation functions.

• nested syntax impedes the functional programing paradigm of function
chaining, esp when each function has 2 or more arguments (e.g. map).

here's a short summary of the nesting problem:

(map f x) ; 1 level of chaining
(map g (map f x)) ; 2 levels
(map h (map g (map f x))) ; 3 levels

compare:

x | f | g | h ----> unix pipe
x // f // g // h ----> Mathematica
h @ g @ f @ x ----> Mathematica
x.f.g.h -------> various OOP langs, esp Ruby, javascript
h g f x -------> some functional langs, Haskell, Ocaml

The way the above works is that each of f, g, h is a lambda themselves
that maps. (that is, something like “(lambda (y) (map f y))”)

Note, that any of the f, g, h may be complex pure functions (aka
lambda). Because in lisp, each lambda itself will in general have
quite a lot nested parens (which cannot be avoided), so this makes any
chaining of functions of 2 args, for more than 2 or 3 levels of
nesting, unusable for practical coding. One must define the functions
separately and just call their names, or use function composition with
lambda (which gets complex quickly). One major aspect of this problem
is that the scope of vars becomes hard to understand in the deep
nested source code. This is worse in elisp, because emacs is
dynamically scoped, so you have to avoid using var of same name.

More detail here:

the section “The Cons Business” at

• The Concepts and Confusions of Prefix, Infix, Postfix and Fully
Nested Notations
http://xahlee.org/UnixResource_dir/writ/notations.html

Your input will be highly valued. Though, forgive me to say, that i
think my opinion on this is beyond any computer scientist of any
standing can try to tell me otherwise. If they disagree (which i think
most of them won't), i think their knowledge and experience in the
area of computer languages and syntax and notations, IQ, are inferior
to me.

Xah
http://xahlee.org/


Kaz Kylheku

unread,
Jan 19, 2009, 9:30:35 PM1/19/09
to
On 2009-01-19, Daniel Weinreb <d...@alum.mit.edu> wrote:
> Now, here's how the Common Lisp looks next to the Ruby:
>
> (mp (readlines "blob.txt") (lambda (line) (mp (split line) #'to-i)))
> IO.readlines("blob.txt").map{|line| line.split.map{|s| s.to_i }}

Why use #'to-i?

You're up against an unoptimized, slow implementation/language that has no
compiler. So it's fair to go through symbolic lookup via 'to-i.

There, you've saved one more character.

Xah Lee

unread,
Jan 20, 2009, 11:35:22 PM1/20/09
to
Xah Lee wrote:
> > consider code produced by corporations, as opposed to with respect to
> > some academic or philsophical logical analysis. Looked in another way,
> > consider if we can compile stat of all existing pyhton code used in
> > real world, you'll find the above style is rarely used.

Rhodri James wrote:
> I *was* thinking of code produced in the real world, and I don't buy
> your assertion. I'm not an academic, and I wouldn't hesitate to lay
> down a line of code like that. As I said before, it fits into English
> language idioms naturally, and as a result is pretty self-descriptive.

The issue is whether the python code in the real world, by statistics,
uses a style such as list comprehension as discussed in this thread.
(to simplify the matter: whether code out there uses list
comprehension in situations when it can, by what percentage. I claimed
it is rare, being borderline esoteric. This can be interpreted as less
that 10%)

In partcular, the issue, is not about your opinion or joe tech
geeker's personal experiences of what they have seen. In newsgroups,
every joe geeker makes claims using personal experiences as if that
apply for the world. I, of course also based on my claims on personal
experience, however, the difference is that my claim is explicitly
made in the context of applying to the world. For example, my claim is
not about my experiences being such and such. My claim is about such
and such is so in the real world.

If, now, you claim that it is not so, then perhaps we might have
something to argue about.

If you can, say, as a example, have a code that crawl the web of all
google's own python code at google code for example, and do some
simple analysis and report what is the percentage, and if that
percentage is more than what i claim, i'll find it very interesting.
But if you simply have such code to do such scale of analysis, that's
far more interesting by itself than our debate.

Similarly, if you can find any evidence, say by some code researcher's
reports, that'd be great. At this point, i recall that i have read
books on such report. You might try to do research on such books and
read up.

> Long experience particularly in C suggests that you are entirely wrong...

Try to study more. I recommend spend less time coding or tech geeking
(such as reading blogs, studying languages, or studying computer
science ). I recommend taking a course in community college on
philosophy, logic, literature, economics. After that, your thinking in
coding and all the social issues related to coding, and computing
industry, will sharpen by far.

Xah
http://xahlee.org/


Xah Lee

unread,
Jan 20, 2009, 11:43:33 PM1/20/09
to
On Jan 20, 7:51 pm, Terry Reedy <tjre...@udel.edu> wrote:
> Rhodri James wrote:
> > Computer languages are not human languages, but computer language
> > constructs do attempt to map onto human language constructs to
> > provide some measure of comprehensibility. Where a construct like
> > list comprehension maps very well onto idiomatic English, dismissing
> > it as "ad hoc syntax soup" is just plain wrong.
>
> Especially given that Python's comprehensions are taken from Haskell's
> which are taken from mathematics' set builder notation. Also, care was
> taken to have a correspondance between comprehension syntax and nested
> for- and if- statement syntax. Someone recently made a proposal that
> would break that correspondance and which would be 'ad hoc' and it will
> surely be rejected.

I suggest you take a course in math history. This is a pratical
suggestion. There are community colleges perhaps near you. Register
and take a course. If you are in USA, usually they are very cheap
because it is funded by taxes. Typically it's 3 hours or so a week,
adding homework it might be 10 per week, and last about 3 months. Try
it. It's fun.

The above won't actually teach you much in htis issue. To get some
level of basic knowledge on this, you'll have to have few years on
inter-displinary study associated math history, math notations, markup
langs, computer syntax, mathematical linguistics related to grammar n
syntax...

Being a tech geeking hip creature, perhaps you'll never take the above
advice. You'd rather immediate read some slashdot or google something
and post profusely in online forums. For this, i recommend a few
articles of my own:

• The Codification of Mathematics
http://xahlee.org/cmaci/notation/math_codify.html

• The TeX Pestilence
http://xahlee.org/cmaci/notation/TeX_pestilence.html

• The Problems of Traditional Math Notation
http://xahlee.org/cmaci/notation/trad_math_notation.html

• A Notation for Plane Geometry
http://xahlee.org/cmaci/notation/plane_geometry_notation.html

• The Concepts and Confusions of Prefix, Infix, Postfix and Fully
Nested Notations
http://xahlee.org/UnixResource_dir/writ/notations.html

of course, i also have very technical and practical book length
tutorials on emacs lisp, python, perl, java, javascript/html/css, php,
povray. You can find them on my website.

Thanks.

Xah
http://xahlee.org/


alex23

unread,
Jan 20, 2009, 11:49:09 PM1/20/09
to
On Jan 21, 2:35 pm, Xah Lee <xah...@gmail.com> wrote:
> I, of course also based on my claims on personal
> experience, however, the difference is that my claim is explicitly
> made in the context of applying to the world. For example, my claim is
> not about my experiences being such and such. My claim is about such
> and such is so in the real world.

So the claims of people who have personally worked on "real world"
code that contains list comprehensions isn't valid, whereas you who
have _no_ such experience have a more genuine understanding of the
world? Which is somehow totally unbiased and yet somehow always agrees
with your preconceived notions?

Talk about Aristotle and his wife's teeth...

> I recommend taking a course in community college on
> philosophy, logic, literature, economics. After that, your thinking in
> coding and all the social issues related to coding, and computing
> industry, will sharpen by far.

Because clearly what the world needs is more of your style of academic
understanding over practical, pragmatic knowledge. And arrogant
condescension...we sure can't get enough of that.

Xah Lee

unread,
Jan 21, 2009, 4:13:03 AM1/21/09
to
Rhodri James wrote:
> I recommend spending less time being certain that you are correct
> without seeking evidence

I don't concur.

For instance, when you are talking to a bunch of kids, you have to be
sure of yourself, else they run all over you, even if they didn't mean
to be rude.

Also, one's demeanor must commensurate one's knowledge. If i pamper
you, you might think i'm a whimp, and run on with your opinions and
thoughts unbridled, which, can be considered as a miscommunication on
my part.

Xah
http://xahlee.org/


William James

unread,
Feb 22, 2009, 3:13:11 AM2/22/09
to
André Thieme wrote:

> (map #(map (fn [s] (Integer/parseInt s)) (.split % "\\s")) (line-seq
> (reader "blob.txt")))

An error results:

java.lang.Exception: Unable to resolve symbol: reader in this context

This works:

(map #(map (fn [s] (Integer/parseInt s)) (.split % "\\s"))

(.split (slurp "junk") "\r?\n"))

André Thieme

unread,
Feb 22, 2009, 12:16:46 PM2/22/09
to
William James schrieb:

Download clojure.contrib and do first:
(use 'clojure.contrib.duck-streams)

The clojure-contrib.jar needs to be in your classpath, then the (use ..)
will work.

The function reader can read files, but you can also put in http, https
or ftp, if your document waits there (and not locally).


André
--
Lisp is not dead. It’s just the URL that has changed:
http://clojure.org/

Hubert

unread,
Feb 24, 2009, 6:56:01 AM2/24/09
to
On 20 Jan., 00:10, Daniel Weinreb <d...@alum.mit.edu> wrote:
> Drew Krause wrote:
> > OK, I want to create a nestedlistin Lisp (always of only integers) from a
> > text file...

How is this recursive version?
It uses a read macro, too, which treats #\Newline as a terminating
macro character
returning NIL.
Empty lines are parsed as ().
An empty file is parsed as (), too.

(defun read-list-from-line (input-stream
&optional result
&aux (obj (read input-stream nil 'eof)))
(cond
((eq obj 'eof)
(return-from read-list-from-line 'eof))
((null obj)
(return-from read-list-from-line (nreverse result)))
(t
(push obj result)
(read-list-from-line input-stream result))))

(defun read-list-from-stream (input-stream
&optional result
&aux (line (read-list-from-line input-stream)))
(cond
((eq 'eof line)
(return-from read-list-from-stream (nreverse result)))
(t
(push line result)
(read-list-from-stream input-stream result))))

(defun make-list-from-file (filename)
(with-open-file (input-stream filename)
(let ((*readtable* (copy-readtable)))
(set-macro-character #\Newline
#'(lambda (st ch) (declare (ignore st ch))))
(read-list-from-stream input-stream))))

nick_keigh...@hotmail.com

unread,
Feb 24, 2009, 10:00:41 AM2/24/09
to
On 17 Jan, 17:16, Xah Lee <xah...@gmail.com> wrote:
> comp.lang.lisp,comp.lang.scheme,comp.lang.functional,comp.lang.python,comp.­lang.ruby
>
> Here's a interesting toy problem posted by Drew Krause to
> comp.lang.lisp:
>
> ------------------------
> On Jan 16, 2:29 pm, Drew Krause wrote [paraphrased a bit]:
>
> OK, I want to create a nested list in Lisp (always of only integers)

> from a text file, such that each line in the text file would be
> represented as a sublist in the 'imported' list.
>
> example of a file's content
>
> 3 10 2
> 4 1
> 11 18
>
> example of programing behavior

> (make-list-from-text "blob.txt") => ((3 10 2) (4 1) (11 18))
>
> -----------------
> Here's a emacs lisp version:

>
> (defun read-lines (file)
> "Return a list of lines in FILE."
> (with-temp-buffer
> (insert-file-contents file)
> (split-string
> (buffer-substring-no-properties 1 (point-max)) "\n" t)
> )
> )
>
> (defvar mylist '() "result list" )
> (setq mylist '()) ; init in case eval'd again
>
> (mapc
> (lambda (x) (setq mylist
> (cons (split-string x " ") mylist )) )
> (read-lines "xxblob.txt")
> )
>
> The above coding style is a typical maintainable elisp.
>
> In a show-off context, it can be reduced to by about 50%, but still
> far verbose than ruby or say perl (which is 1 or 2 lines. (python
> would be 3 or 5)).
>
> Note that the result element is string, not numbers. There's no easy
> way to convert them on the fly. 3 or so more lines will be needed to
> do that.
>
> The “read-lines” function really should be a built-in part of elisp.

> It is not so mostly because emacs people have drawn themselves into a
> elitist corner, closing off to the outside world. (i am sending a
> suggestion to the official emacs dev list on adding read-line now.)

scheme:

(define (read-line port)
(define (rec-read-line port line)
(define next-char (peek-char port))
(define number #f)
(cond ((eof-object? next-char) '())
((char=? next-char #\newline) (read-char port) line)
((char-numeric? next-char)
(set! number (read port))
(cons number (rec-read-line port line)))
((char=? next-char #\space)
(read-char port)
(rec-read-line port line))
(else (error (string-append "bad character \""
(string next-char) "\"")))
))

(rec-read-line port '())
)

(define (make-int-list port)
(define line (read-line port))
(if (null? line)
'()
(cons line (make-int-list port))))

(define (make-list-from-text file)
(make-int-list (open-input-file file)))

> -----------------


>
> w_a_x_...@yahoo.com gave a ruby solution:
>
> IO.readlines("blob.txt").map{|line| line.split }
>
> augumented by Jilliam James for result to be numbers:
>

> IO.readlines("blob.txt").map{|line| line.split.map{|s| s.to_i }}
>

> Very beautiful.
>
> -------------------
>
> That's really the beauty of Ruby.
>
> This problem and ruby code illustrates 2 fundamental problems of lisp,
> namely, the cons problem, and the nested syntax pain. Both of which
> are practically unfixable.
>

> The lisp's cons fundamentally makes nested list a pain to work with.
> Lisp's nested syntax makes functional sequencing cumbersome.
>
> In the ruby code, its post-fix sequential notation (as a side effect
> of its OOP notation) brings out the beauty of functional sequencing
> paradigm (sometimes known as functional chain, sequencing, filtering,
> unix piping).
>
> its list, like all modern high level langs such as perl, php, python,
> javascript, don't have the lisp's cons problem. The cons destroys the
> usability of lists up-front, untill you have some at least 2 full-time
> years of coding lisp to utilize cons properly. (and even after that,
> it is still a pain to work with, and all you gain is a bit of speed
> optimization in rare cases that requires largish data, most of which
> has better solutions such as a database.)
>
> Both of these problems i've published articles on.
>
> For more detail on the cons problem, see


> the section “The Cons Business” at
>
> • Fundamental Problems of Lisp
> http://xahlee.org/UnixResource_dir/writ/lisp_problems.html
>

> For more detail on the nested syntax problem for function chaining,
> see
> the section “How Purely Nested Notation Limits The Language's Utility”
> at:

nick_keigh...@hotmail.com

unread,
Feb 25, 2009, 6:24:07 AM2/25/09
to
On 24 Feb, 15:00, nick_keighley_nos...@hotmail.com wrote:
> On 17 Jan, 17:16, Xah Lee <xah...@gmail.com> wrote:

> > Here's a interesting toy problem posted by Drew Krause to
> > comp.lang.lisp:
>
> > ------------------------
> > On Jan 16, 2:29 pm, Drew Krause wrote [paraphrased a bit]:
>
> > OK, I want to create a nested list in Lisp (always of only integers)
> > from a text file, such that each line in the text file would be
> > represented as a sublist in the 'imported' list.
>
> > example of a file's content
>
> > 3 10 2
> > 4 1
> > 11 18
>
> > example of programing behavior
> > (make-list-from-text "blob.txt") => ((3 10 2) (4 1) (11 18))

<snip>

an assignment-free version

(define (read-line port)
(define (rec-read-line port line)

(let ((next-char (peek-char port)))


(cond ((eof-object? next-char) '())
((char=? next-char #\newline) (read-char port) line)
((char-numeric? next-char)

(let ((number (read port)))
(cons number (rec-read-line port line))))


((char=? next-char #\space)
(read-char port)
(rec-read-line port line))
(else (error (string-append "bad character \""

(string next-char) "\""))))))

(rec-read-line port '()))

(define (make-int-list port)
(let ((line (read-line port)))
(if (null? line)
'()
(cons line (make-int-list port)))))

(define (make-list-from-text file)
(make-int-list (open-input-file file)))

(define (mk) (make-list-from-text "blob.txt"))

<snip>

number was originally define'd but I discovered there were
limits to where you could define things

nick_keigh...@hotmail.com

unread,
Feb 25, 2009, 6:34:33 AM2/25/09
to
On 17 Jan, 17:16, Xah Lee <xah...@gmail.com> wrote:
> comp.lang.lisp,comp.lang.scheme,comp.lang.functional,comp.lang.python,comp.­lang.ruby

<snip>

> The lisp's cons fundamentally makes nested list a pain to work with.
> Lisp's nested syntax makes functional sequencing cumbersome.

so hide it

(define (make-list stream eos?)
(let ((atom (stream)))
(if (eos? atom)
'()
(cons atom (make-list stream eos?)))))

(define (make-int-list port)
(make-list (lambda () (read-line port)) null?))

the nasty cons then only appears in a single function which
you can hide in a library


> In the ruby code, its post-fix sequential notation (as a side effect
> of its OOP notation) brings out the beauty of functional sequencing
> paradigm (sometimes known as functional chain, sequencing, filtering,
> unix piping).
>
> its list, like all modern high level langs such as perl, php, python,
> javascript, don't have the lisp's cons problem. The cons destroys the
> usability of lists up-front, untill you have some at least 2 full-time
> years of coding lisp to utilize cons properly. (and even after that,
> it is still a pain to work with, and all you gain is a bit of speed
> optimization in rare cases that requires largish data, most of which
> has better solutions such as a database.)

is my code to build a list that bad?


> Both of these problems i've published articles on.
>
> For more detail on the cons problem, see
> the section “The Cons Business” at
>
> • Fundamental Problems of Lisp
>  http://xahlee.org/UnixResource_dir/writ/lisp_problems.html

I read it. Your point seems to be "cons becomes difficult
with deeply nested structures". Could you give an example?

Xah Lee

unread,
Feb 25, 2009, 1:18:17 PM2/25/09
to
On Feb 25, 3:34 am, nick_keighley_nos...@hotmail.com wrote:
> the nasty cons then only appears in a single function which
> you can hide in a library

I think the following answers that.

Q: If you don't like cons, lisp has arrays and hashmaps, too.

A: Suppose there's a lang called gisp. In gisp, there's cons but also
fons. Fons are just like cons except it has 3 cells with 3 accessors:
car, cbr, cdr. Now, gisp is a old lang, the fons are deeply rooted in
the lang. Every some 100 lines of code you'll see a use of fons with
its extra accessor cbr, or any one of the cbaar, cdabr, cbbar, cbbbar,
etc. You got annoyed by this. You as a critic, complains that fons is
bad. But then some gisp fanatics retorts: “If you don't like fons,
gisp has cons, too!”.

You see, by “having something too”, does not solve the problem of
pollution. Sure, you can use just cons in gisp, but every lib or
other's code you encounter, there's a invasion of fons with its cbar,
cbbar, cbbbar. The problem created by fons does not go away by “having
cons too”.

above is from

---------

> I read it. Your point seems to be "cons becomes difficult
> with deeply nested structures". Could you give an example?

There are few examples in these articles:

• The Concepts and Confusions of Prefix, Infix, Postfix and Fully
Nested Notations
http://xahlee.org/UnixResource_dir/writ/notations.html

the above, 3rd section, gives detail about the problems of fully
nested syntax. In particular, it shows a source code snippet of
language with fully nested syntax, but is not lisp, so that lispers
can get a fresh impression.

the above, is a concrete example of showing how full nesting is
cumbersome, by constrasting a simple program in Ruby and lisp.

• Why Lisp Do Not Have A Generic Copy-List Function
http://xahlee.org/UnixResource_dir/writ/lisp_equal_copy_list.html

the above, shows the cons problem, by looking Kent Pitman's article
with a different perspective.

A short Plain Text Excerpt of the ruby article cited above follows.
------------------------------

compare:

Xah
http://xahlee.org/

Michael Weber

unread,
Feb 25, 2009, 2:42:04 PM2/25/09
to
On Jan 16, 9:29 pm, Drew Krause <drkra...@mindspring.com> wrote:
> OK, I want to create a nested list in Lisp (always of only integers) from a
> text file, such that each line in the text file would be represented as a
> sublist in the 'imported' list.

Here's a solution using SERIES:

(defpackage #:text
(:use #:cl #:series #:split-sequence)
(:export #:make-list-from-text))

(in-package #:text)

(eval-when (:compile-toplevel :load-toplevel :execute)
(series::install :implicit-map t))

(defun make-list-from-text (filename)
(let* ((lines (scan-file filename #'read-line))
(split (split-sequence #\space lines
:remove-empty-subseqs t)))
(collect (mapcar #'parse-integer split))))

#||
(make-list-from-text "/tmp/blob.txt")
==> ((3 10 2) (4 1) (11 18))
||#

grande...@gmail.com

unread,
Feb 25, 2009, 3:42:09 PM2/25/09
to
with collect and collect:

(with-open-file (f "blob.txt" :direction :input)
(loop for linea = (read-line f nil nil) while linea
collect
(with-input-from-string (s linea)
(loop for entero = (read s nil nil) while entero
collect entero))))

Xah Lee

unread,
Feb 25, 2009, 7:46:52 PM2/25/09
to


Here's a actual lisp code. I don't consider it readable, due to the
profusion of parens.

(defun lisp-complete-symbol (&optional predicate)
"Perform completion on Lisp symbol preceding point.
Compare that symbol against the known Lisp symbols.
If no characters can be completed, display a list of possible
completions.
Repeating the command at that point scrolls the list.

When called from a program, optional arg PREDICATE is a predicate
determining which symbols are considered, e.g. `commandp'.
If PREDICATE is nil, the context determines which symbols are
considered. If the symbol starts just after an open-parenthesis, only
symbols with function definitions are considered. Otherwise, all
symbols with function definitions, values or properties are
considered."
(interactive)
(let ((window (get-buffer-window "*Completions*" 0)))
(if (and (eq last-command this-command)
window (window-live-p window) (window-buffer window)
(buffer-name (window-buffer window)))
;; If this command was repeated, and
;; there's a fresh completion window with a live buffer,
;; and this command is repeated, scroll that window.
(with-current-buffer (window-buffer window)
(if (pos-visible-in-window-p (point-max) window)
(set-window-start window (point-min))
(save-selected-window
(select-window window)
(scroll-up))))

;; Do completion.
(let* ((end (point))
(beg (with-syntax-table emacs-lisp-mode-syntax-table
(save-excursion
(backward-sexp 1)
(while (= (char-syntax (following-char)) ?\')
(forward-char 1))
(point))))
(pattern (buffer-substring-no-properties beg end))
(predicate
(or predicate
(save-excursion
(goto-char beg)
(if (not (eq (char-before) ?\())
(lambda (sym) ;why not just nil ? -sm
(or (boundp sym) (fboundp sym)
(symbol-plist sym)))
;; Looks like a funcall position. Let's double check.
(if (condition-case nil
(progn (up-list -2) (forward-char 1)
(eq (char-after) ?\())
(error nil))
;; If the first element of the parent list is an open
;; parenthesis we are probably not in a funcall position.
;; Maybe a `let' varlist or something.
nil
;; Else, we assume that a function name is expected.
'fboundp)))))
(completion (try-completion pattern obarray predicate)))
(cond ((eq completion t))
((null completion)
(message "Can't find completion for \"%s\"" pattern)
(ding))
((not (string= pattern completion))
(delete-region beg end)
(insert completion)
;; Don't leave around a completions buffer that's out of date.
(let ((win (get-buffer-window "*Completions*" 0)))
(if win (with-selected-window win (bury-buffer)))))
(t
(let ((minibuf-is-in-use
(eq (minibuffer-window) (selected-window))))
(unless minibuf-is-in-use
(message "Making completion list..."))
(let ((list (all-completions pattern obarray predicate)))
(setq list (sort list 'string<))
(or (eq predicate 'fboundp)
(let (new)
(while list
(setq new (cons (if (fboundp (intern (car list)))
(list (car list) " <f>")
(car list))
new))
(setq list (cdr list)))
(setq list (nreverse new))))
(if (> (length list) 1)
(with-output-to-temp-buffer "*Completions*"
(display-completion-list list pattern))
;; Don't leave around a completions buffer that's
;; out of date.
(let ((win (get-buffer-window "*Completions*" 0)))
(if win (with-selected-window win (bury-buffer))))))
(unless minibuf-is-in-use
(message "Making completion list...%s" "done")))))))))

Xah
http://xahlee.org/


0 new messages