Yet another 'when to use macros' question....
Jonathon McKitrick
re-written most of them to be generated by macros.
I've heard the adage to only use macros when they are needed, but
technically, since only Lisp has this type of source-generating macro,
they are never *really* needed, because the same problem can be solved
in some other language without macros.
So, when I sit back and look at code that is more concise, but a bit
less transparent, when do I decide that macros are or are not a good
idea?
Pascal Bourguignon
You need macros when you want to write something that you cannot write
in any other language: CONTROL STRUCTURES!
For example, you want to write:
(in-parallel
(loop for i from 0 to 10 do (print i) (sleep 1))
(loop for i from 10 to 20 do (print i) (sleep 1))
(print :start)
(progn (sleep 10) (print :stop)))
So you write this macro:
(defmacro in-parallel (&rest forms)
`(progn
,@(mapcar (lambda (form) `(when (zerop (linux:fork)) ,form
(ext:exit)))
(butlast forms))
,@(last forms)))
Then:
[2]> (in-parallel
(loop for i from 0 to 10 do (print i) (sleep 1))
(loop for i from 10 to 20 do (print i) (sleep 1))
(print :start)
(progn (sleep 10) (print :stop)))
0
10
:START
1
11
12
2
13
3
14
4
15
5
16
6
17
7
18
8
19
9
:STOP
:STOP
[3]>
20
10
--
__Pascal Bourguignon__ http://www.informatimago.com/
What is this talk of 'release'? Klingons do not make software 'releases'.
Our software 'escapes' leaving a bloody trail of designers and quality
assurance people in it's wake.
Wade Humeniuk
The best way to learn is to look at some well written code. Take some
time and look at some asdf packages on cliki.net. Take your time and study
macro use in actual complex code.
As an example, defparser that comes with LW has a lex/yacc like grammar
that makes defining the parser less error prone and more expressive.
The macro becomes a domain specific language. (another parser is cl-yacc
which Juliusz (the author) uses for a C parser)
Here is an example of parsing time
RUNNING-LOG 2 > (parse-time "12:30:59.2")
4505920
(:S 59.2 :M 30 :H 12)
RUNNING-LOG 3 > (parse-time "12h30m59.2s")
4505920
(:H 12 :M 30 :S 59.2)
RUNNING-LOG 4 >
and the parser defined by defparser....
(defparser time-parser
((time type-of-time) $1)
((type-of-time :error) (error "Invald Time Format"))
((type-of-time colon-time) (loop for tc in '(:s :m :h)
for tv in (reverse $1)
collect tc
collect tv))
((type-of-time hms-time) $1)
((num int) $1)
((num float) $1)
((colon-time num) (list $1))
((colon-time num \:) (list $1 0))
((colon-time \:) (list 0 0))
((colon-time \: colon-time) (if (= (length $2) 3)
(error "Too Many Values in COLON TIME")
(cons 0 $2)))
((colon-time num \: colon-time) (if (= (length $3) 3)
(error "Too Many Values In COLON TIME")
(cons $1 $3)))
((tc s) :s)
((tc m) :m)
((tc h) :h)
((hms-time num tc) (list $2 $1))
((hms-time num tc num) (case $2
(:s (error "Invalid HMS Time Format"))
(:m (list :m $1 :s $3))
(:h (list :h $1 :m $3))))
((hms-time num tc hms-time) (if (member $2 $3)
(error "Duplicate Time Component in HMS Time")
(cons $2 (cons $1 $3)))))
(defun parse-time (string &optional (error-p nil))
(handler-case
(let ((v (with-input-from-string (stream string)
(time-parser (lambda () (time-lexer stream))))))
(values (apply #'time-to-hundreths v) v))
(error (c) (if error-p (error c)
(values nil nil)))))
The macroexpansion of time-parser is .... (which can be
very error prone if done by hand for each parser)
(PROGN
(DEFUN TIME-PARSER
(#:G545 &OPTIONAL (#:G546 #'IDENTITY) &KEY
(PARSERGEN::DEFAULT-ERROR-RECOVERY T)
(PARSERGEN::MESSAGE-STREAM T))
(DECLARE (HARLEQUIN-COMMON-LISP:LAMBDA-LIST
PARSERGEN::LEXER-FUNCTION
&OPTIONAL
PARSERGEN::IDENTITY-FUNCTION
&KEY
PARSERGEN::DEFAULT-ERROR-RECOVERY
PARSERGEN::MESSAGE-STREAM))
(LET ((PARSERGEN::*LEXER* #:G545)
(PARSERGEN::*SYMBOL-TO-STRING* #:G546)
(PARSERGEN::*GOTO-TABLE*
(LOAD-TIME-VALUE (PARSERGEN::BUILD-GOTO-TABLE
'((0
. #(TYPE-OF-TIME
7
HMS-TIME
6
NUM
5
\:
4
INT
3
FLOAT
2
COLON-TIME
1))
(14 . #(\: 11))
(5
. #(S 12 \: 11 M 10 H 9 TC 8))
(4
. #(NUM
14
\:
4
INT
3
FLOAT
2
COLON-TIME
15))
(11
. #(NUM
14
\:
4
INT
3
FLOAT
2
COLON-TIME
13))
(16 . #(S 12 M 10 H 9 TC 8))
(8
. #(HMS-TIME
17
NUM
16
INT
3
FLOAT
2))))))
(PARSERGEN::*ACTION-TABLE*
'#(#(-5 #(\:) -3 #(FLOAT) -4 #(INT) 0 #(:ERROR))
#(1 #(:EOI)) #(4 #(:EOI \: S M H))
#(3 #(:EOI \: S M H))
#(7 #(:EOI) -3 #(FLOAT) -4 #(INT) -5 #(\:))
#(5 #(:EOI) -12 #(\:) -10 #(H) -11 #(M) -13
#(S))
#(2 #(:EOI)) #(:ACCEPT #(:EOI))
#(13 #(:EOI) -3 #(FLOAT) -4 #(INT))
#(12 #(:EOI INT FLOAT)) #(11 #(:EOI INT FLOAT))
#(6 #(:EOI) -3 #(FLOAT) -4 #(INT) -5 #(\:))
#(10 #(:EOI INT FLOAT)) #(9 #(:EOI))
#(5 #(:EOI) -12 #(\:)) #(8 #(:EOI))
#(-10 #(H) -11 #(M) -13 #(S) 14 #(:EOI))
#(15 #(:EOI))))
(PARSERGEN::*ACTION-FUNCTION-TABLE*
'#(TIME-PARSER-ACTION0 TIME-PARSER-ACTION1
TIME-PARSER-ACTION0 TIME-PARSER-ACTION0
TIME-PARSER-ACTION0 TIME-PARSER-ACTION5
TIME-PARSER-ACTION6 TIME-PARSER-ACTION7
TIME-PARSER-ACTION8 TIME-PARSER-ACTION9
TIME-PARSER-ACTION10 TIME-PARSER-ACTION11
TIME-PARSER-ACTION12 TIME-PARSER-ACTION13
TIME-PARSER-ACTION14 TIME-PARSER-ACTION15))
(PARSERGEN::*ACTION-NARGS-TABLE*
'#(1 1 1 1 1 1 2 1 2 3 1 1 1 2 3 3))
(PARSERGEN::*ACTION-NT-TABLE*
'#(TIME TYPE-OF-TIME TYPE-OF-TIME NUM NUM
COLON-TIME COLON-TIME COLON-TIME COLON-TIME
COLON-TIME TC TC TC HMS-TIME HMS-TIME HMS-TIME))
(PARSERGEN::*ERROR-PRODUCTIONS* '#((:ERROR)))
(PARSERGEN::*ERROR-ACTION-FUNCTION-TABLE*
'#(TIME-PARSER-ERROR-ACTION-0))
(PARSERGEN::*ERROR-ACTION-NT-TABLE*
'#(TYPE-OF-TIME))
(PARSERGEN::*ACCEPT-WITHOUT-EOI-P* NIL)
(PARSERGEN::*ERROR-LOCATION-INFORMATION-HOOK* NIL)
(PARSERGEN::*DEFAULT-ERROR-RECOVERY*
PARSERGEN::DEFAULT-ERROR-RECOVERY)
(PARSERGEN::*PARSER-MESSAGE-STREAM*
PARSERGEN::MESSAGE-STREAM))
(DECLARE (SPECIAL
PARSERGEN::*LEXER*
PARSERGEN::*SYMBOL-TO-STRING*
PARSERGEN::*ACTION-TABLE*
PARSERGEN::*GOTO-TABLE*
PARSERGEN::*ACTION-FUNCTION-TABLE*
PARSERGEN::*ACTION-NARGS-TABLE*
PARSERGEN::*ACTION-NT-TABLE*
PARSERGEN::*ERROR-ACTION-FUNCTION-TABLE*
PARSERGEN::*ERROR-ACTION-NT-TABLE*
PARSERGEN::*DEFAULT-ERROR-RECOVERY*
PARSERGEN::*PARSER-MESSAGE-STREAM*))
(RUN-PARSER)))
(DEFUN TIME-PARSER-ACTION0 ($1) (DECLARE (IGNORABLE $1)) $1)
(DEFUN TIME-PARSER-ACTION1 ($1)
(DECLARE (IGNORABLE $1))
(LOOP FOR TC IN '(:S :M :H)
FOR TV IN (REVERSE $1)
COLLECT TC
COLLECT TV))
(DEFUN TIME-PARSER-ACTION5 ($1)
(DECLARE (IGNORABLE $1))
(LIST $1))
(DEFUN TIME-PARSER-ACTION6 ($1 $2)
(DECLARE (IGNORABLE $1 $2))
(LIST $1 0))
(DEFUN TIME-PARSER-ACTION7 ($1)
(DECLARE (IGNORABLE $1))
(LIST 0 0))
(DEFUN TIME-PARSER-ACTION8 ($1 $2)
(DECLARE (IGNORABLE $1 $2))
(IF (= (LENGTH $2) 3)
(ERROR "Too Many Values in COLON TIME")
(CONS 0 $2)))
(DEFUN TIME-PARSER-ACTION9 ($1 $2 $3)
(DECLARE (IGNORABLE $1 $2 $3))
(IF (= (LENGTH $3) 3)
(ERROR "Too Many Values In COLON TIME")
(CONS $1 $3)))
(DEFUN TIME-PARSER-ACTION10 ($1)
(DECLARE (IGNORABLE $1))
:S)
(DEFUN TIME-PARSER-ACTION11 ($1)
(DECLARE (IGNORABLE $1))
:M)
(DEFUN TIME-PARSER-ACTION12 ($1)
(DECLARE (IGNORABLE $1))
:H)
(DEFUN TIME-PARSER-ACTION13 ($1 $2)
(DECLARE (IGNORABLE $1 $2))
(LIST $2 $1))
(DEFUN TIME-PARSER-ACTION14 ($1 $2 $3)
(DECLARE (IGNORABLE $1 $2 $3))
(CASE $2
(:S (ERROR "Invalid HMS Time Format"))
(:M (LIST :M $1 :S $3))
(:H (LIST :H $1 :M $3))))
(DEFUN TIME-PARSER-ACTION15 ($1 $2 $3)
(DECLARE (IGNORABLE $1 $2 $3))
(IF (MEMBER $2 $3)
(ERROR "Duplicate Time Component in HMS Time")
(CONS $2 (CONS $1 $3))))
(DEFUN TIME-PARSER-ERROR-ACTION-0 ()
(ERROR "Invald Time Format")))
Wade
Kenny Tilton
My2: never decide about something until you know you are doing it right.
If your code looks less transparent, you either used macros where you
should not have or used them poorly. Or both, I guess. As a newby, you
need to worry especially about having cocked things up. It happens.
You did not say much about this use case, so it is hard to comment. One
thing I notice: (background first) in an extreme case (such as DEFCLASS)
it is normal for one macro form to expand into multiple methods, as you
say you are doing. We do this when the similarity is among sets of
methods. So over here I see a methods A1 B1 C1 covering some serious
functionality, and over here I see A2 B2 C2. a little different but
cloneable from each other. Then I might say, well let's make a macro
DEF-Abstraction-over-ABC-HANDLER providing one place the things that
will tell you how to write methods An BN Cn automatically for you.
If you are not doing anything that hairy (ie, building an API or
language within your application) then I am concerned that the
similarity has you writing a macro to write three different methods. I
would rather see the similarity you see in the bodies of the (still
three) functions expressed as a macro each function will invoke.
kenny
Coby Beck
news:iLWDf.10484$cj3....@news-wrt-01.rdc-nyc.rr.com...
> Jonathon McKitrick wrote:
>> I have several methods and functions that are similar, and I have
>> re-written most of them to be generated by macros.
>>
>> I've heard the adage to only use macros when they are needed, but
>> technically, since only Lisp has this type of source-generating macro,
>> they are never *really* needed, because the same problem can be solved
>> in some other language without macros.
>>
>> So, when I sit back and look at code that is more concise, but a bit
>> less transparent, when do I decide that macros are or are not a good
>> idea?
>>
>
> My2: never decide about something until you know you are doing it right.
>
> If your code looks less transparent, you either used macros where you
> should not have or used them poorly. Or both, I guess. As a newby, you
> need to worry especially about having cocked things up. It happens.
I took "less transparent" to mean "hiding implementation", which together
with a generous interpretation of concise makes it sound like a good thing.
--
Coby Beck
(remove #\Space "coby 101 @ bigpond . com")
Pascal Costanza
It would be helpful to see some of your code to be able to give you more
specific advise.
Pascal
--
My website: http://p-cos.net
Closer to MOP & ContextL:
http://common-lisp.net/project/closer/
Jonathon McKitrick
> It would be helpful to see some of your code to be able to give you more
> specific advise.
I am generating 2 types of xy plots with cl-pdf. Each method
instantiates and returns a pdf:plot-xy, created with mostly the same
arguments, but a few important ones are different. It's a long list of
keyword args and values.
(defmacro define-plot (plot-type title max-x nb-ticks label-position
label-rotation label-font-size format-string)
"Define a method to generate a pdf plot."
`(defmethod ,plot-type ((rpt <birkman>) x y plot-data)
(make-instance 'pdf:plot-xy
:x x :y y :width *chart-dx* :height *chart-dy*
:title ,title
:line-width 1
:post-draw-chart-fn #'post-draw-chart
:series (mapcar #'car plot-data)
:labels&colors (mapcar #'cadr plot-data)
:point-radius 2
:x-axis-options '(:min-value 0 :max-value ,max-x
:nb-ticks ,nb-ticks
:label-position ,label-position
:label-rotation ,label-rotation
:label-font-size ,label-font-size
:format-string ,format-string)
:y-axis-options '(:min-value 0 :max-value 100)
:legend-options '(:label-font-size 7.0))))
(define-plot plot-interests "Interests" 10 10 :right 0 8
"~/report::format-interest-label/")
(define-plot plot-components "Components" 21 21 nil 45 7
"~/report::format-component-label/")
The original version had a method for each of the plot types to return
the correct object.
The original question is: Does it make more sense to have 2 methods or
1 macro? The 2 methods mean more code to be maintained. The 1 macro
is less code, but less readable and/or clear about what is going on,
perhaps. And maybe a macro is just total overkill here.
Pascal Costanza
To me the macro looks good. It captures a repetitive piece of code, and
it seems to me that the use of define-plot leads to clearer code than
the more wordy defmethod form. The define-plot also has a good
documentation and understandable parameter names. I don't see anything
to worry about...
Majorinc
use...@informatimago.com says...
> You need macros when you want to write something that you cannot write
> in any other language: CONTROL STRUCTURES!
Ouch ... check Unicon, for example - you can write control
structures there just like functions, without any macros, i.e.
syntactic manipulation of source code.
This is excelent example how even experienced programmers in
Lisp do not understand its advantages and disadvantages.
Majorinc
@o13g2000cwo.googlegroups.com>, j_mck...@bigfoot.com says...
> I have several methods and functions that are similar, and I have
> re-written most of them to be generated by macros.
>
> I've heard the adage to only use macros when they are needed, but
> technically, since only Lisp has this type of source-generating macro,
> they are never *really* needed, because the same problem can be solved
> in some other language without macros.
From my point of view, macros are the worst thing that happened
to Lisp, because they compromise the most important feature of
Lisp, code=data, especially the ability of programs to generate
and execute other programs dynamically.
Imagine program that produces and execute million small and
fast programs in a minute - it is much better that these small,
generated programs contain only function calls. If these small,
generated program contain macros, you'll have millions of macro
expansions in a minute in RUNTIME, and it can turn otherwise
feasible concept into nightmare ...
Kenny Tilton
I have not finished my coffee yet... why not make that a function?
That said, contra Keene, it is not wise to hide make-instance. Maybe
make the class smarter with a good initialize-instance (or
shared-initialize-instance in the extreme case) do the computations you
are having the macro write to compute initarg values (such as
x-axis-options).
Are you going to have ten of these plots by the time you get done? If
not, the macro is not worth the trouble. With separate methods it does
not matter if they start to diverge, you have two wodges of code to
playt with.
Finally, go ahead and use keyword args so you do not end up with strings
of unlabelled values like ... 21 21 nil 45 7...
You can combine this because the initialize methods &allow-other-keys,
so you do not even have to give the plot class the slots for what are
now parameters to your macro.
kt
M Jared Finder
Please give an example. How would you write something like dolist in
Unicon?
-- MJF
M Jared Finder
Huh??
Why is calling a macro so much slower than calling a function?
And why is the macroexpansion happening at runtime?
-- MJF
joseph...@gmail.com
Majorinc wrote:
> Imagine program that produces and execute million small and
> fast programs in a minute - it is much better that these small,
> generated programs contain only function calls. If these small,
> generated program contain macros, you'll have millions of macro
> expansions in a minute in RUNTIME, and it can turn otherwise
> feasible concept into nightmare ...
Have you actually had this happen in a real example? Because in this
century most of us are using compilers for our Lisp programs, and the
macro expansions happen at compile-time.
The final code performance does not slow down; if anything, our macros
allow us to extend the compiler to produce *better* code by allowing
high-level description of low-level optimizations.
Perhaps in the "good" old days, when Lisp interpreters still roamed the
Earth and had different semantics from the compilers, you might have
been right. That's before my time.
André Thieme
> That said, contra Keene, it is not wise to hide make-instance.
I have not read her book.. What did Keene say about make-instance? How
and why should one hide it and why do you argue against it (and in what
situations)?
André
--
Pascal Costanza
Majorinc talk about generating programs dynamically. If dynamically
created code contains macro invocations, they have to be expanded first.
Of course the solution if this becomes a problem is to avoid generating
code that uses macro invocations. It's probably a good idea to think
about using functional abstraction so that no code is generated and
compiled at runtime.
Majorinc
ja...@hpalace.com says...
>
> Please give an example. How would you write something like dolist in
> Unicon?
You already have very powerful construct for that,
(dolist (i L) ... ) <=> every i:=!L do { .... }
Generally, "programmer defined control operations" are not
simple, they use concept of coe-xpression. The best place to
inform about it is "The Icon Programming Language, Third
Edition" chapter "Co-expressions" or "Icon Analyst 55", all
available on the net, google knows. An example of something
that could be called do-reverse-list in Lisp is included.
Kenny Tilton
> Kenny Tilton schrieb:
>
>
>>That said, contra Keene, it is not wise to hide make-instance.
>
>
> I have not read her book.. What did Keene say about make-instance?
She said hide it. :) In a make-widget call that calls:
(make-instance 'widget)
> How
> and why should one hide it ...
Never.
> ... and why do you argue against it (and in what
> situations)?
Because hiding it forces one to learn the API implicit in make-widget.
Now I cannot just look at the widget class and it's initialize method, I
have to study make-widget. Which will not have the flexibility of
make-instance (if it does, what is the point?). Then you want to
subclass widget and do make-instance 'son-of-widget and it does not work
because Necessary Stuff was being done in make-widget. etc etc
kenny
Pascal Costanza
As always, all rules have exceptions. To put it differently:
"You had something to hide
Should have hidden it, shouldn’t you
Now you’re not satisfied
With what you’re being put through
[...]
Now you’re standing there tongue tied
You’d better learn your lesson well
Hide what you have to hide
And tell what you have to tell"
jayessay
> In article <zfidnRNLYaq...@speakeasy.net>,
> ja...@hpalace.com says...
>
>
> >
> > Please give an example. How would you write something like dolist in
> > Unicon?
>
> You already have very powerful construct for that,
>
> (dolist (i L) ... ) <=> every i:=!L do { .... }
That's irrelevant. You should still be able to write it in the
language and have it look and work just like "every i:=!L do { .... }"
FYI dolist itself is a macro.
/Jon
--
'j' - a n t h o n y at romeo/charley/november com
jayessay
> talk about generating programs dynamically. If dynamically
> created code contains macro invocations, they have to be expanded
> first.
Yes, but then you just compile them. This can make sense (and vastly
improve performance) even if you only use the result once.
> Of course the solution if this becomes a problem is to avoid
> generating code that uses macro invocations. It's probably a good idea
> to think about using functional abstraction so that no code is
> generated and compiled at runtime.
That's a good "rule of thumb" to always keep in mind, but in the cases
where I've "needed" to generate code that has macro invocations in it,
compiling has eliminated the "issue" raised here.
Lastly, it is probably worth noting that there are some
implementations where the compiler is always invoked (no traditional
interpreter involved), so the expansion only happens once even without
explicit (compile ....)
joseph...@gmail.com
Maybe I'm just fuzzy on the whole thing, but isn't this simply the
difference between coding an interpreter and coding a compiler? (Where
the implementation language/target language is Lisp, of course).
If things are being generated dynamically, it is simply whether the
generated objects are built out of Lisp functions already compiled to
machine code, or Lisp s-expressions passed (hopefully only once) to the
(potentially slow) Lisp compiler, or whether they are data structures
interpreted (once, or potentially many times) at some higher level (by
a potentially even slower) interpreter?
That is the real issue, where the distinction between Lisp functions
and Lisp macros is really meaningless: if a "regular" function is slow
or a macro expander function is slow, you want to do it as few times as
possible, and if you can do it ahead of time, do so.
To condemn macros because interpreters are slow is blaming the raw
material for the decision of the architect.
Coby Beck
news:MPG.1e4b047f5...@news.carnet.hr...
> In article <1138755721.025782.194260
> @o13g2000cwo.googlegroups.com>, j_mck...@bigfoot.com says...
>> I have several methods and functions that are similar, and I have
>> re-written most of them to be generated by macros.
>>
>> I've heard the adage to only use macros when they are needed, but
>> technically, since only Lisp has this type of source-generating macro,
>> they are never *really* needed, because the same problem can be solved
>> in some other language without macros.
>
> From my point of view, macros are the worst thing that happened
> to Lisp, because they compromise the most important feature of
> Lisp, code=data, especially the ability of programs to generate
> and execute other programs dynamically.
Any newbies would be wise to ignore all advice and opinion this fellow
offers here, his ignorance is matched only by his confidence.
> Imagine program that produces and execute million small and
> fast programs in a minute - it is much better that these small,
> generated programs contain only function calls. If these small,
> generated program contain macros, you'll have millions of macro
> expansions in a minute in RUNTIME, and it can turn otherwise
> feasible concept into nightmare ...
Only the most foolish of carpenters will throw out his screwdriver because
it does not drive nails. If you are programmatically generating code and
macro-expansion is costing you time then obviously don't generate macro
forms, generate the code directly which is all a macro will do for you
anyway. That said, a macro-expander is in fact just a code generator so the
line between time spent in your own code generating what you will pass to
the compiler and time spent by the implementation's macro expansion code
generator is slightly more obscur than extremely blurry.
It kind of boggles the mind to see someone call anything, much less the most
innovative and powerful tool lisp offers, a bad thing simply by imagining a
situation in which its use would be inappropriate. "Doctor, it hurts when I
crack my knuckles." Majorinc's answer: "Let's amputate your fingers."
Jonathon McKitrick
Kenny Tilton wrote:
> Finally, go ahead and use keyword args so you do not end up with strings
> of unlabelled values like ... 21 21 nil 45 7...
>
> You can combine this because the initialize methods &allow-other-keys,
> so you do not even have to give the plot class the slots for what are
> now parameters to your macro.
I'm trying to figure out one tricky part. The initialize-instance
method of a class I have sub-classed creates additional internal
objects. The plot creates instances of axis, legend, etc. Initially,
the options are passed a keyword list argument. But I want to force
them to have those values when the plot object is created without
having to set them manually. So creating an instance of plot-small
will, when internally creating its legend, will use font-small, or
something like that.
Do I do this in the initializer :before, :after, elsewhere, or none of
the above?
Peter Seibel
I'm not sure I understand your question, but to the extent I do, I
think you'll be fine if you just do your initialization in an
initialize-instance :after method. Since :after methods run in
most-specific-last order, your method will run after the methods
specialized on the superclass. Thus any initialization code that is
provided by the author of the class you are subclassing should run
before your initialize-instance method and thus all the object's slots
that would be initialized if you just made an instance of the
superclass will be initialized by the time your code sees it.
-Peter
--
Peter Seibel * pe...@gigamonkeys.com
Gigamonkeys Consulting * http://www.gigamonkeys.com/
Practical Common Lisp * http://www.gigamonkeys.com/book/
Thomas A. Russ
> From my point of view, macros are the worst thing that happened
> to Lisp, because they compromise the most important feature of
> Lisp, code=data, especially the ability of programs to generate
> and execute other programs dynamically.
I disagree. Macros don't compromise the code=data paradigm, they
exploit it. In fact, macros are, by definition, code generators, so it
seems that they really are where most of the action is in the code
equals data arena.
The whole point about it being easy to define domain-specific languages
in lisp is based on the ability of macros to extend the language
seamlessly.
> Imagine program that produces and execute million small and
> fast programs in a minute - it is much better that these small,
> generated programs contain only function calls. If these small,
> generated program contain macros, you'll have millions of macro
> expansions in a minute in RUNTIME, and it can turn otherwise
> feasible concept into nightmare ...
This really has nothing to do with macros. That's because the code=data
paradigm only works at the surface level. Compiled code is not
something that is typically generated by user-level program generators.
That is the job of the compiler. So it doesn't really matter much if
you generate S-Expressions which invoke functions or macros.
If you run the resulting generated program with EVAL, you wil be doing
evaluation and it isn't at all clear that adding macroexpansion into the
mix would make much difference. On the other hand, if you compile the
resulting code, for example, using the COMPILE function, then all of the
macroexpansion happens during compilation and not at runtime.
So this then becomes a design question of whether the faster execution
of compiled code saves enough time to be worth the time spent in the
compiler. Clearly if you generate a program (more technically a
function) and want to run it many times, the compilation step will be
worthwhile. For a single execution, it isn't clear what would be
faster.
(And yes, there are also several lisp systems where it doesn't make any
real difference for one-off programs because the interpreter compiles
any complicated form anyway....)
--
Thomas A. Russ, USC/Information Sciences Institute
Peter Seibel
Ah, I just saw one of your follow-up questions on the cl-pdf mailing
list and now I understand that you want, in your subclass, to provide
default initialization arguments for the initargs used by the
superclass initialization methods. In that case, you want to look at
the :DEFAULT-INITARGS option to DEFCLASS.
Jonathon McKitrick
> I'm not sure I understand your question, but to the extent I do, I
> think you'll be fine if you just do your initialization in an
> initialize-instance :after method. Since :after methods run in
> most-specific-last order, your method will run after the methods
> specialized on the superclass. Thus any initialization code that is
> provided by the author of the class you are subclassing should run
> before your initialize-instance method and thus all the object's slots
> that would be initialized if you just made an instance of the
> superclass will be initialized by the time your code sees it.
That's exactly the problem. The internal objects are already
initialized by the time I get to the :after method. I need to force
values to them that are normally passed as keyword parameters. Since
the keyword parameters are lists, and then passed to the inner objects
for calculations, by the time I get to :after, the initialization I
want to change has already occurred.
P.S. Have the book, love it.
Jonathon McKitrick
Peter Seibel wrote:
> the :DEFAULT-INITARGS option to DEFCLASS.
That seems close... but I can't quite get it to work. I want this:
(make-instance '<interest-chart>
:x x :y y
:series (mapcar #'car plot-data)
:labels&colors (mapcar #'cadr plot-data)
:x-axis-options '(:min-value 0 :max-value 10
:nb-ticks 10
:label-position :right ; right of x-axis point to be labeled
:label-font-size 8.0
:format-string "~/report::format-interest-label/")
:y-axis-options '(:min-value 0 :max-value 100)
; :legend-options '(:label-font-size 7.0)
))
to work like this:
(defclass <interest-chart> (<base-chart>)
()
(:default-initargs legend-options '(:label-font-size 18.0)))
So that rather than sending the options with make-instance, they will
already be set. But somehow it's not working....
Edi Weitz
Maybe because of the missing colon...
BTW, are you sure you want literal constants there?
--
Lisp is not dead, it just smells funny.
Real email: (replace (subseq "spam...@agharta.de" 5) "edi")
Jonathon McKitrick
Edi Weitz wrote:
> Maybe because of the missing colon...
Bingo! I thought I was supposed to use the slot name rather than the
slot keyword name.
> BTW, are you sure you want literal constants there?
Yes, I want to force a font size. Were you talking about using
variables for the size?
Edi Weitz
> Bingo! I thought I was supposed to use the slot name rather than
> the slot keyword name.
No, it's not a "slot keyword name" but an initarg - and it's called
"default initargs" for a reason... :)
Initargs don't have to be keyword symbols, BTW, this is just a
convention.
CL-USER 1 > (defclass foo () ((x :initarg x)))
#<STANDARD-CLASS FOO 2068636C>
CL-USER 2 > (describe (make-instance 'foo 'x 42))
#<FOO 20692B4C> is a FOO
X 42
>> BTW, are you sure you want literal constants there?
>
> Yes, I want to force a font size. Were you talking about using
> variables for the size?
No, I was talking about /literal/ constants, something like
'(a b c)
vs.
(list 'a 'b 'c)
You should only use the former if you are sure your code will /never/
modify it. See [3-13] here:
<http://www.faqs.org/faqs/lisp-faq/part3/>
Cheers,
Edi.
Jonathon McKitrick
Kenny Tilton wrote:
> Finally, go ahead and use keyword args so you do not end up with strings
> of unlabelled values like ... 21 21 nil 45 7...
>
> You can combine this because the initialize methods &allow-other-keys,
> so you do not even have to give the plot class the slots for what are
> now parameters to your macro.
I'm a bit foggy on how this is done. What do you mean in the second
paragraph? Would you give an example?
Kenny Tilton
Jonathon McKitrick wrote:
> Kenny Tilton wrote:
>
>>Finally, go ahead and use keyword args so you do not end up with strings
>>of unlabelled values like ... 21 21 nil 45 7...
>>
>>You can combine this because the initialize methods &allow-other-keys,
>>so you do not even have to give the plot class the slots for what are
>>now parameters to your macro.
>
>
> I'm trying to figure out one tricky part. The initialize-instance
> method of a class I have sub-classed creates additional internal
> objects.
I should have mentioned that I noticed that and it worries me. It does
make it hard to author the resulting interface, because you are trying
to fabricate all these things at once, so you will be losing control
over how the internal objects get authored.
Overall, you have bitten off a lot as a newbie, and are creating
problems for yourself with your own design choices.
One thing you should do is have, as just one example, a slot for
x-axis. That normally gets populated by an initializer, but if you
supply one, boom, end of story. Your initializer just has to say (unless
x-axis....).
OK, that guarantees you complete flexibility. Now about overriding
superclass behavior. Hard to do with after methods, as you have
discovered. I would take an approach like this:
If the value supplied for x-axis is an instance, leave it alone.
If it is a symbol:
(setf (x-axis self) (make-instance (x-axis self) :plot self))
Two things here:
(1) Let the class of the x-axis worry about initializing its own ass
(2) yes, the plot must have attributes sufficient to tell the x-axis how
to implement itself, unless:
If the value supplied for x-axis is a list, you:
(setf (x-axis self) (apply 'make-instance (car (x-axis self))
:plot self
(cdr (x-axis self))))
[Just typing here so parens may not align etc]
The idea is that you stay out of the way and let make-instance work,
even to the extent of allowing the make-instance of internal instances
to be programmed by the surrounding code.
(I think I recall you already doing something like this for the axes in
the macro version.)
Another trick is:
If the value supplied for x-axis is a function, you:
(setf (x-axis self) (funcall (x-axis self) self))
Now you are having all sorts of fun, with a closure that can do anything
including use arbitray closed-over variables. This is the best way to
avoid loading up the xy-plot with a kazillion attributes for all the
internal instances.
One final note. One trick for handling the subclassing problem as a
whole would be to have initialize-instance :after methods simply massage
the parameterization in the x-axis slot, then have an i-i :around method
that looks roughly like:
(defmethod ii :around ((self xy-plot) &rest initargs)
(call-next-method) ;; let all :after methods run
(here-and-only-here-make-an-axis))
But I kinda like the idea of just having xy-plot have the ii :after
method and letting it make whatever x-axis you need, using all the
tricks I outlined for specifying x-axis (instance, class name, class
name and args, closure factory) to easily author your x-axis.
kenny
Kenny Tilton
I was a little foggy too and elsewhere, kinda busy and just writing
enough to encourage more questions. &allow-other-keys was a bit of a
headfake. What I wanted to say is that you can get more info into an
initialize-instance method than just slot initargs, by specifying
additonal keys:
(defclass thingy ()
())
(defclass transaktion ()
((units-of-work :initform nil :accessor uow))) ;; ugh, bad name!!
(defmethod initialize-instance :after ((self thingy) &rest iargs &key tx)
(assert tx () "Dude, you cannot make thingy ~a outside a
transaction." self)
(push self (uow tx)))
(make-instance 'thingy)
-> Error: Dude, you cannot make thingy #<THINGY> outside a transaction.
(make-instance 'thingy :tx (make-instance 'transaktion))
-> #<THINGY @ #x21b72dd2>
And you cannot specify just any key, such as TXX (two Xs):
(make-instance 'thingy :txx (make-instance 'transaktion))
-> Error: :TXX is an invalid initarg to make-instance of class
#<STANDARD-CLASS THINGY>. The valid initargs are :TX.
kenny
Majorinc
nos...@foo.com says...
> > You already have very powerful construct for that,
> >
> > (dolist (i L) ... ) <=> every i:=!L do { .... }
>
> That's irrelevant. You should still be able to write it in the
> language and have it look and work just like "every i:=!L do { .... }"
>
Just read the rest of the post.
Majorinc
@g14g2000cwa.googlegroups.com>, joseph...@gmail.com says...
>
> Maybe I'm just fuzzy on the whole thing, but isn't this simply the
> difference between coding an interpreter and coding a compiler? (Where
> the implementation language/target language is Lisp, of course).
>
> If things are being generated dynamically, it is simply whether the
> generated objects are built out of Lisp functions already compiled to
> machine code, or Lisp s-expressions passed (hopefully only once) to the
> (potentially slow) Lisp compiler, or whether they are data structures
> interpreted (once, or potentially many times) at some higher level (by
> a potentially even slower) interpreter?
It is the problem of the macro expansion time, and issue of
compiler/interpreter has some merit, but it is really not
essential. Essence is in macro expanding time that has no
equivalent in functional solution at all.
If you use macros on the way similar to C++, macro expansion
time is not the problem - macro is expanded once, and expanded
code is executed many times. For somehow extreme example,some
time ago I used to preprocess code in this fashion:
10 i=i+1
20 print i
30 i=i+1
40 print i
...
9980 print i
9990 goto 10
because it is quite a bit faster than
10 i=i+1
20 print i
30 goto 10
But, if you dynamically generate programs, and you execute
these programs once, then *macro expansion time* has to be
added to the macro execution time and the trick turns against
you. What was very time efficient in one context, now is
extremely unefficient.
Compiling does not help.
>
> That is the real issue, where the distinction between Lisp functions
> and Lisp macros is really meaningless: if a "regular" function is slow
> or a macro expander function is slow, you want to do it as few times as
> possible, and if you can do it ahead of time, do so.
Yes, you are partly right - you only need to draw the
conclusion bit further. Because, in general case, you are not
satisfied to avoid slow expansions and execution times, you
want to effectively minimize both of them.
Execution time cannot be eliminated, only thing you can do is
to use best algorithms and write the fastest code you can. And
it is hard job.
On the other side, macro expanding time can be eliminated
completely and easily - just define functions instead of macros
whenever you can.
It has lot of sense, I think.
M Jared Finder
(For those who don't want to search on Google, see
<http://www.cs.arizona.edu/icon/lb3.htm>.
Okay, so Icon encourages you to use generators for your looping
construct. Can generators be implemented in Icon without generators?
Cause they can in Lisp, see the Series package.
<http://series.sourceforge.net/>.
-- MJF
Thomas A. Russ
> Yes, you are partly right - you only need to draw the
> conclusion bit further. Because, in general case, you are not
> satisfied to avoid slow expansions and execution times, you
> want to effectively minimize both of them.
>
> Execution time cannot be eliminated, only thing you can do is
> to use best algorithms and write the fastest code you can. And
> it is hard job.
>
> On the other side, macro expanding time can be eliminated
> completely and easily - just define functions instead of macros
> whenever you can.
Not necessarily. You may be just pushing the macro expansion time to
another part of your program. In other words, why does it make a
difference if you use macros and have the expansion done by the compiler
or interpreter (BTW, there are some interpreters that only macro-expand
any form once), or if you do equivalent work in your program generating
code? In the latter case, all you've done is moved the runtime work
from the lisp system into your own code. All you've done is written
your own macro expansion routine.
"just define functions instead of macros whenever you can", however, is
generally good advice. Unless you need some aspect of macros, functions
would be the preferred implementation anyway.
BTW, do you actually have any empirical evidence that macro expansion is
slowing down your generated code? Or is this all just speculation based
on some impression that macro expansion has to be slow?
jayessay
Just post the code for writing your own version of "every i:=!L do {
.... }" that looks, works and fits into the language just like the
native one. If you can't do that your claim is simply wrong.
jayessay
> Majorinc wrote:
> > In article <zfidnRNLYaq...@speakeasy.net>, ja...@hpalace.com
> > says...
> >
> >> Please give an example. How would you write something like dolist
> >> in Unicon?
> > You already have very powerful construct for that, (dolist (i L)
> > ... ) <=> every i:=!L do { .... }
> > Generally, "programmer defined control operations" are not simple,
> > they use concept of coe-xpression. The best place to inform about it
> > is "The Icon Programming Language, Third Edition" chapter
> > "Co-expressions" or "Icon Analyst 55", all available on the net,
> > google knows. An example of something that could be called
> > do-reverse-list in Lisp is included.
>
> (For those who don't want to search on Google, see
> <http://www.cs.arizona.edu/icon/lb3.htm>.
Icon is a neat language; I used it and liked it many years ago. But
co-expressions have nothing to do with what macros provide or the
ability to create any, let alone arbitrary, sorts of new _language
level_ constructs, be they control constructs or whatever.
> Okay, so Icon encourages you to use generators for your looping
> construct. Can generators be implemented in Icon without generators?
I don't recall that being possible at the time, but things may have
changed. Isn't this the very sort of thing this Majorinc guy is
supposed to show by providing a concrete example? You should also be
able to do things like write the Icon "Procedure" and "while"
constructs this way. I don't believe this is doable. Of course, if
he shows the code for it, then that would prove otherwise.
> Cause they can in Lisp, see the Series
> package. <http://series.sourceforge.net/>.
See also the development in SICP.
Majorinc
ja...@hpalace.com says...
>
> Okay, so Icon encourages you to use generators for your looping
> construct. Can generators be implemented in Icon without generators?
> Cause they can in Lisp, see the Series package.
> <http://series.sourceforge.net/>.
Hey, I do not claim that Icons control operations are *better*
(neither they are worse) than Lisp's. It is way too extensive
question. I Just mentioned other languages beside Lisp that I
used and that had programmer-defined control operations ...
But if you asked, yes, it is possible and relatively simple to
define generators in the majority of the languages. Basically,
one need to write functions so they "remember" what happened
last time they are called "in the same context" and to continue
where they stopped before ...
Majorinc
cb...@mercury.bc.ca says...
> That said, a macro-expander is in fact just a code generator so the
> line between time spent in your own code generating what you will pass to
> the compiler and time spent by the implementation's macro expansion code
> generator is slightly more obscur than extremely blurry.
You are partly right. Macros are *special* case of code
generators - one that each time expands on the place it is
executed.
If you generate 1000 macros, they'll expand code to be executed
1000 times. In general case, you can do better than that.
>
> It kind of boggles the mind to see someone call anything, much less the most
> innovative and powerful tool lisp offers, a bad thing simply by imagining a
> situation in which its use would be inappropriate. "Doctor, it hurts when I
> crack my knuckles." Majorinc's answer: "Let's amputate your fingers."
Maybe what I'm telling is - dont buy cheap shoes, they can work
for some time, but in long terms you'll certainly need better
tools and you'll regret you bought cheap ones ...
Try to look this into perspective - one can develop his
software for say, 2-3 years and live quite happily with macros.
Readable, fast code ... But if his problem mature to the point
he need generated code, macros will show their ugly, expanding
face ...
The love for macros in the Lisp community is even more
surprising, because from, say, AI point of view, development of
the programming methods that are not well suited for programs
that write programs has quite a limited sense ...
Coby Beck
> In article <ia9Ef.160076$AP5.160063@edtnps84>,
> cb...@mercury.bc.ca says...
>
>> That said, a macro-expander is in fact just a code generator so the
>> line between time spent in your own code generating what you will pass to
>> the compiler and time spent by the implementation's macro expansion code
>> generator is slightly more obscur than extremely blurry.
>
> You are partly right. Macros are *special* case of code
> generators - one that each time expands on the place it is
> executed.
No they don't. Macro's expand at macro-expansion time which does not have
to happen at runtime. This is a highly unusual situation, macro-expanding
at runtime. Even if you are generating code at runtime you can expand the
macro once by compiling it and then run it 1000 times.
> If you generate 1000 macros, they'll expand code to be executed
> 1000 times. In general case, you can do better than that.
Once again, you missed the point. If a macro is an inappropriate tool for
your particular problem, then don't use it. So yes, you can do better that
that, and you can do it in Lisp as it is designed without removing macros
entirely from the language.
>> It kind of boggles the mind to see someone call anything, much less the
>> most
>> innovative and powerful tool lisp offers, a bad thing simply by imagining
>> a
>> situation in which its use would be inappropriate. "Doctor, it hurts
>> when I
>> crack my knuckles." Majorinc's answer: "Let's amputate your fingers."
>
> Maybe what I'm telling is - dont buy cheap shoes, they can work
> for some time, but in long terms you'll certainly need better
> tools and you'll regret you bought cheap ones ...
No, this is not what you are saying. You are saying that your shoes are too
tight and instead of losening the laces, you want to cut them off.
> Try to look this into perspective - one can develop his
> software for say, 2-3 years and live quite happily with macros.
> Readable, fast code ... But if his problem mature to the point
> he need generated code, macros will show their ugly, expanding
> face ...
Problems do not "mature" to the point of needing generated code. Generating
code at runtime is a fairly special case and hardly appropriate for most
problems, no matter how smart and mature you get. And again, you are
forgetting that a macro *is* a code generator, one that typically runs at
compile time. But if you are working on a problem that requires generating
code, you have the choice of what code to generate and you can certainly
avoid generating macros. But again, as I mentioned before and another
poster pointed out, this just means doing the extra code generation on your
own rather than using the implementation's macro expander. So by removing
macros from the language to the detriment of everyone else, because it is
possible for you to misues them, you may not have even saved anything at
runtime.
> The love for macros in the Lisp community is even more
> surprising, because from, say, AI point of view, development of
> the programming methods that are not well suited for programs
> that write programs has quite a limited sense ...
I'm not sure what that means.
Majorinc
says...
> Majorinc, Kazimir <fa...@email.address> writes:
> >
> > On the other side, macro expanding time can be eliminated
> > completely and easily - just define functions instead of macros
> > whenever you can.
>
> Not necessarily. You may be just pushing the macro expansion time to
> another part of your program.
Well, I spoke only about function vs macros here. However, I
can answer, there are some tricks ... for example, instead of
expanding large macro each time it has to be evaluated in the
code, one can maintain code equivalent to expanded one as data
in the memory and insert it (possibly with small changes) in
the right place. Inserting is faster than expanding.
But, my point is somewhere else - macros are everywhere in
Lisp, and although they look harmless, their macroexpansion is
so slow that they are practically useless in generated code.
I'll demonstrate it with macro PUSH.
>
> BTW, do you actually have any empirical evidence that macro expansion is
> slowing down your generated code? Or is this all just speculation based
> on some impression that macro expansion has to be slow?
I have some experience with expansion of the code in assembler,
and I know it is generally acceptable only if code is expanded
once, and executed many times. Typical for C, not for Lisp.
Here is the test for PUSH.
[1]> (defun pump-l(n x)(setf L ())(dotimes(i n)(push x L)))
[2]> (defun eval-whole-L()(dolist (f L)(eval f)))
[3]> (pump-l 100000 '(progn (setq A ())(push 0 A)))
[4]> (time (eval-whole-l))
Real time: 3.046875 sec.
Run time: 3.046875 sec.
Space: 91965272 Bytes
GC: 144, GC time: 0.375 sec.
_________________________________________
Now, look what happens without macroexpansion:
_________________________________________
[5]> (pump-l 100000 '(progn (setq A ())(setq A (cons 0 A))))
[6]> (time (eval-whole-l))
Real time: 0.1875 sec.
Run time: 0.1875 sec.
Space: 800008 Bytes
GC: 1, GC time: 0.015625 sec.
_________________________________________
Speed up of two orders of value (only 25% time is spent in
(setq A (cons 0 A)) part.
So, PUSH (and other macros) in general case - should not be
used in the generated code. I think that now everone can agree
about that.
Only thing that we might disagree is importance of that fact.
From my point of view, it is huge, disasterous problem - the
code generation is not something marginal for Lisp, it is its
essential feature. If macros should not be used *in generated
code*, there is no good reason to invest any resources in their
development.
Pascal Bourguignon
Of course nobody reaches your conclusion: it's wrong. To wit:
(defun gen-expr (n x) (loop repeat n collect x))
(dolist (basexp '((push 0 a) (setq a (cons 0 a))))
(terpri) (print basexp)
(time (eval `(let ((f (compile nil
(lambda ()
(let ((a '()))
,@(gen-expr 10000 basexp))))))
(loop repeat 10000 do (funcall f))))))
(PUSH 0 A)
Real time: 36.10611 sec.
Run time: 24.32 sec.
Space: 827559032 Bytes
GC: 323, GC time: 13.9 sec.
(SETQ A (CONS 0 A))
Real time: 42.884426 sec.
Run time: 28.41 sec.
Space: 808599008 Bytes
GC: 330, GC time: 19.16 sec.
But still you don't need eval:
(dolist (basexp '((push 0 a) (setq a (cons 0 a))))
(terpri) (print basexp)
(time (let ((f (compile nil `(lambda ()
(let ((a '()))
,@(gen-expr 10000 basexp))))))
(loop repeat 10000 do (funcall f)))))
(PUSH 0 A)
Real time: 37.99427 sec.
Run time: 25.89 sec.
Space: 818101168 Bytes
GC: 747, GC time: 15.93 sec.
(SETQ A (CONS 0 A))
Real time: 45.871666 sec.
Run time: 25.27 sec.
Space: 808437528 Bytes
GC: 742, GC time: 15.8 sec.
There's no significant difference.
> Only thing that we might disagree is importance of that fact.
> From my point of view, it is huge, disasterous problem - the
> code generation is not something marginal for Lisp, it is its
> essential feature. If macros should not be used *in generated
> code*, there is no good reason to invest any resources in their
> development.
Note that there is strictly no difference if you write:
(dolist (basexp '((push 0 a) (setq a (cons 0 a))))
(terpri) (print basexp)
(time (let ((f (compile nil
(print `(lambda ()
(let ((a '()))
,@(gen-expr 10 (macroexpand basexp))))))))
(loop repeat 10000 do (funcall f)))))
(PUSH 0 A)
(LAMBDA NIL
(LET ((A 'NIL)) (SETQ A (CONS 0 A)) (SETQ A (CONS 0 A)) (SETQ A (CONS 0 A))
(SETQ A (CONS 0 A)) (SETQ A (CONS 0 A)) (SETQ A (CONS 0 A))
(SETQ A (CONS 0 A)) (SETQ A (CONS 0 A)) (SETQ A (CONS 0 A))
(SETQ A (CONS 0 A))))
Real time: 0.169301 sec.
Run time: 0.08 sec.
Space: 839904 Bytes
GC: 1, GC time: 0.02 sec.
(SETQ A (CONS 0 A))
(LAMBDA NIL
(LET ((A 'NIL)) (SETQ A (CONS 0 A)) (SETQ A (CONS 0 A)) (SETQ A (CONS 0 A))
(SETQ A (CONS 0 A)) (SETQ A (CONS 0 A)) (SETQ A (CONS 0 A))
(SETQ A (CONS 0 A)) (SETQ A (CONS 0 A)) (SETQ A (CONS 0 A))
(SETQ A (CONS 0 A))))
Real time: 0.243853 sec.
Run time: 0.08 sec.
Space: 838936 Bytes
GC: 1, GC time: 0.02 sec.
Which shows you that you can use macros all you want.
--
__Pascal Bourguignon__ http://www.informatimago.com/
The rule for today:
Touch my tail, I shred your hand.
New rule tomorrow.
Edi Weitz
ROTFL!
This is not an answer to you directly because your recent behaviour in
this newsgroup disqualifies you from that. But to any "newbies" who
might potentially read this: It's pure nonsense, the guy doesn't know
what he's talking about.
bradb
because the functions are not compiled (assuming that he is not running
on a compiled only Lisp like SBCL)?
As a newbie, I wouldn't mind knowing the mistakes that Major is making.
Cheers
Brad
Wade Humeniuk
And even the more unsophisticated code runs the same
(dolist (basexp '((push 0 a) (setq a (cons 0 a))))
(setf a nil)
(terpri) (print basexp)
(time (loop repeat 10000 do (eval `(funcall ,(compile nil `(lambda () ,basexp)))))))
(PUSH 0 A)
Timing the evaluation of (LOOP REPEAT 10000 DO (EVAL (SYSTEM::BQ-LIST (QUOTE FUNCALL)
(COMPILE NIL (SYSTEM::BQ-LIST (QUOTE LAMBDA) NIL BASEXP)))))
user time = 11.776
system time = 0.010
Elapsed time = 0:00:12
Allocation = 194084576 bytes standard / 38308666 bytes conses
0 Page faults
Calls to %EVAL 40000
(SETQ A (CONS 0 A))
Timing the evaluation of (LOOP REPEAT 10000 DO (EVAL (SYSTEM::BQ-LIST (QUOTE FUNCALL)
(COMPILE NIL (SYSTEM::BQ-LIST (QUOTE LAMBDA) NIL BASEXP)))))
user time = 12.437
system time = 0.000
Elapsed time = 0:00:12
Allocation = 193298584 bytes standard / 37622123 bytes conses
0 Page faults
Calls to %EVAL 40000
Majorinc
cb...@mercury.bc.ca says...
>
> No, this is not what you are saying. You are saying that your shoes are too
> tight and instead of losening the laces, you want to cut them off.
>
> > Try to look this into perspective - one can develop his
> > software for say, 2-3 years and live quite happily with macros.
> > Readable, fast code ... But if his problem mature to the point
> > he need generated code, macros will show their ugly, expanding
> > face ...
>
> Problems do not "mature" to the point of needing generated code.
Now this is plain bullshit.
Pascal Bourguignon
(defun pump-l(n x)(setf L ())(dotimes(i n)(push x L)))
(defun eval-whole-L()(dolist (f L)(eval f)))
(pump-l 100000 '(progn (setq A ())(push 0 A)))
(time (eval-whole-l))
(pump-l 100000 '(progn (setq A ())(setq A (cons 0 A))))
(time (eval-whole-l))
Well, there are a lot of problem with this code, if it was to be taken
as "production' code. But it's just didling at the REPL, so we won't
take it against Major.
Note that what it is essentially doing is:
(time (loop repeat 1000000 do (eval '(progn (setq a ()) (push 0 a)))))
(time (loop repeat 1000000 do (eval '(progn (setq a ()) (setq a (cons 0 a))))))
He's using clisp. clisp contains an interpreter, so the forms passed
to eval are not compiled. If it was to be execute once, it wouldn't
matter. Since it's in a loop, it's inexcusable not to compile it.
See my answer to Major for how to use COMPILE.
Now, to see the difference between both expressions, you can run:
[59]> (ext:without-package-lock (:cl) (trace push))
;; Tracing macro PUSH.
(PUSH)
[60]> (time (loop repeat 10 do (eval '(progn (setq a ()) (push 0 a)))))
1. Trace: (PUSH 0 A)
1. Trace: PUSH ==> (SETQ A (CONS 0 A))
1. Trace: (PUSH 0 A)
1. Trace: PUSH ==> (SETQ A (CONS 0 A))
1. Trace: (PUSH 0 A)
1. Trace: PUSH ==> (SETQ A (CONS 0 A))
1. Trace: (PUSH 0 A)
1. Trace: PUSH ==> (SETQ A (CONS 0 A))
1. Trace: (PUSH 0 A)
1. Trace: PUSH ==> (SETQ A (CONS 0 A))
1. Trace: (PUSH 0 A)
1. Trace: PUSH ==> (SETQ A (CONS 0 A))
1. Trace: (PUSH 0 A)
1. Trace: PUSH ==> (SETQ A (CONS 0 A))
1. Trace: (PUSH 0 A)
1. Trace: PUSH ==> (SETQ A (CONS 0 A))
1. Trace: (PUSH 0 A)
1. Trace: PUSH ==> (SETQ A (CONS 0 A))
1. Trace: (PUSH 0 A)
1. Trace: PUSH ==> (SETQ A (CONS 0 A))
Real time: 0.001889 sec.
Run time: 0.0 sec.
Space: 53392 Bytes
NIL
[61]> (time (loop repeat 10 do (eval '(progn (setq a ()) (setq a (cons 0 a))))))
Real time: 5.19E-4 sec.
Run time: 0.0 sec.
Space: 4352 Bytes
NIL
[62]> (time (let ((f (compile nil (lambda () (let ((a ())) (push 0 a)))))) (loop repeat 10 do (funcall f))))
1. Trace: (PUSH 0 A)
1. Trace: PUSH ==> (SETQ A (CONS 0 A))
1. Trace: (PUSH 0 A)
1. Trace: PUSH ==> (SETQ A (CONS 0 A))
Real time: 0.001413 sec.
Run time: 0.0 sec.
Space: 17896 Bytes
NIL
[63]> (ext:without-package-lock (:cl) (untrace push))
(PUSH)
--
__Pascal Bourguignon__ http://www.informatimago.com/
This universe shipped by weight, not volume. Some expansion may have
occurred during shipment.
Wade Humeniuk
Your point has some validity, but the problem is the way you have
coded a solution. There are other ways, this simple change to your
version mitigates much of the difference. Like other people have
already commented there are other ways to deal with the issue.
(dolist (basexp '((push 0 a) (setq a (cons 0 a))))
(setf a nil)
(terpri) (print basexp)
(time (loop repeat 100000 do
for expression = (macroexpand basexp)
do (eval expression))))
(PUSH 0 A)
Timing the evaluation of (LOOP REPEAT 100000 DO FOR EXPRESSION = (MACROEXPAND BASEXP) DO
(EVAL EXPRESSION))
user time = 1.892
system time = 0.020
Elapsed time = 0:00:02
Allocation = 10407736 bytes standard / 16512122 bytes conses
0 Page faults
Calls to %EVAL 400000
(SETQ A (CONS 0 A))
Timing the evaluation of (LOOP REPEAT 100000 DO FOR EXPRESSION = (MACROEXPAND BASEXP) DO
(EVAL EXPRESSION))
user time = 1.331
system time = 0.000
Elapsed time = 0:00:02
Allocation = 8802392 bytes standard / 9901023 bytes conses
0 Page faults
Calls to %EVAL 400000
Wade
Majorinc
says...
Edi, please, go home and fuck with your mother instead with me
and "newbies."
bradb
cheating a little & expanding/compiling the closure only once, then
calling it X times? Where Major is expanding/interpreting and
executing the closure X times?
Just saw Wade's version - which does expand/compile/execute the form X
times, and appears to be the same. Cool, I get it I think.
Cheers
Brad
Edi Weitz
> In article <umzh84...@agharta.de>, spam...@agharta.de
> says...
>
>> This is not an answer to you directly because your recent behaviour
>> in this newsgroup disqualifies you from that.
>
> Edi, please, go home and fuck with your mother
Yes, that's what I meant, thanks for confirming that. I'm wondering
why others still insist on feeding you.
bradb
Imagine that you are generating X unique forms, then executing them.
Assuming that the generation cost is the same, then you are better to
generate pre-expanded code rather than macro expand, right?
Thanks for the answers guys, but I am no longer reading this thread.
Major's last response to Edi was beyond all bounds of good taste, or
even poor taste, and I don't want to subject myself to accidentally
reading that kind of electronic pollution again.
Cheers
Brad
Majorinc
says...
> Yes, that's what I meant, thanks for confirming that. I'm wondering
> why others still insist on feeding you.
Oh, I think you know very well how deserved that answer ...
Majorinc
use...@informatimago.com says...
> "bradb" <brad.be...@gmail.com> writes:
>
> > I'm curious as to why Major has the results that he does? Is it
> > because the functions are not compiled (assuming that he is not running
> > on a compiled only Lisp like SBCL)?
> > As a newbie, I wouldn't mind knowing the mistakes that Major is making.
>
>
> (defun pump-l(n x)(setf L ())(dotimes(i n)(push x L)))
> (defun eval-whole-L()(dolist (f L)(eval f)))
> (pump-l 100000 '(progn (setq A ())(push 0 A)))
> (time (eval-whole-l))
> (pump-l 100000 '(progn (setq A ())(setq A (cons 0 A))))
> (time (eval-whole-l))
>
> Well, there are a lot of problem with this code, if it was to be taken
> as "production' code. But it's just didling at the REPL, so we won't
> take it against Major.
>
> Note that what it is essentially doing is:
>
> (time (loop repeat 1000000 do (eval '(progn (setq a ()) (push 0 a)))))
> (time (loop repeat 1000000 do (eval '(progn (setq a ()) (setq a (cons 0 a))))))
>
> He's using clisp. clisp contains an interpreter, so the forms passed
> to eval are not compiled. If it was to be execute once, it wouldn't
> matter. Since it's in a loop, it's inexcusable not to compile it.
> See my answer to Major for how to use COMPILE.
And now, here is my comment on this. Pascal didn't understood
what I am doing at all. He shoot in the dark. I simulated
situation in which these parts of code like
'(progn (setq a ()) (push 0 a))
are generated during runtime, 100 000 of them, and evaluated. I
used 100 000 of same expressions *only because of simplicity*.
In general case, they are - all 100 000 of them - different.
That is the reason I filled them in list, not in loops. They
are *accidentally* same because I simply did not wanted to
write more complex program to test macroexpansion.
However, Pascal didn't understood the point. Instead of that,
he decided to show how "that code" should be rewritten. He
rearranged the code to take an advantage of the *accidental*
fact. He factored out that expression that allowed him to
compile it *once* and execute many times. It is of course,
complete miss my intentions.
Basically, I proved that summation is slow by executing
(dotimes (i 1000) (+ 100 100))
And what Pascal did? Instead of my code he wrote
(setf x (+ 100 100)) (dotimes (i 1000) x)
And he claims he proved something. In my opinion, he simply do
not get it.
Coby Beck
news:1138987744.0...@o13g2000cwo.googlegroups.com...
Yes, he's not compiling. If you need to generate code at and execute it
100,000 times forgetting to compile it is, um, not the very best way to
acheive time efficiency, even if there are no macros.
Which was of course the whole point of most of the answers already given to
him about why he is wrong, I guess he just wanted to provide us very
conclusive evidence that he does not listen to anything he is told.
BTW, if he were doing this on a compile first interpreter likely his results
would be even worse as the form is not only macro-expanded but compiled
100000 times (clever internal optimiaztions not with standing).
Majorinc
@g49g2000cwa.googlegroups.com>, brad.be...@gmail.com
says...
> I imagine Major was simply repeating the same form for simplicity.
> Imagine that you are generating X unique forms, then executing them.
> Assuming that the generation cost is the same, then you are better to
> generate pre-expanded code rather than macro expand, right?
>You are assuming well.
>
> Thanks for the answers guys, but I am no longer reading this thread.
> Major's last response to Edi was beyond all bounds of good taste, or
> even poor taste, and I don't want to subject myself to accidentally
> reading that kind of electronic pollution again.
Another asshole. OK with me.
>
> Cheers
> Brad
>
>
Coby Beck
news:MPG.1e4d9ed43...@news.carnet.hr...
> In article <ymiy80t...@sevak.isi.edu>, t...@sevak.isi.edu
> says...
>
>> Majorinc, Kazimir <fa...@email.address> writes:
>
>> >
>> > On the other side, macro expanding time can be eliminated
>> > completely and easily - just define functions instead of macros
>> > whenever you can.
>>
>> Not necessarily. You may be just pushing the macro expansion time to
>> another part of your program.
>
> Well, I spoke only about function vs macros here. However, I
> can answer, there are some tricks ... for example, instead of
> expanding large macro each time it has to be evaluated in the
> code, one can maintain code equivalent to expanded one as data
> in the memory and insert it (possibly with small changes) in
> the right place. Inserting is faster than expanding.
uh huh, whatever you say...but keep on this path and you will simply be
reinventing the macroexpander you are advising us all is a Bad Thing.
> But, my point is somewhere else - macros are everywhere in
> Lisp, and although they look harmless, their macroexpansion is
> so slow that they are practically useless in generated code.
Not is you compile your generated code, which seems a good idea regardless
of macroexpansion or not.
> I'll demonstrate it with macro PUSH.
>
>
>>
>> BTW, do you actually have any empirical evidence that macro expansion is
>> slowing down your generated code? Or is this all just speculation based
>> on some impression that macro expansion has to be slow?
>
> I have some experience with expansion of the code in assembler,
> and I know it is generally acceptable only if code is expanded
> once, and executed many times. Typical for C, not for Lisp.
Oh yes? You generate and execute C code at runtime? Do show us that. (no
nevermind, it is possible but takes ridiculous contortions, you probably
will see it as equivalent)
> Here is the test for PUSH.
> [1]> (defun pump-l(n x)(setf L ())(dotimes(i n)(push x L)))
Do not create local variables with SETF. Learn about LET. What you are
doing invites all kinds of undefined behaviours including possible
efficiency issues. If it is intended to be special then declare it as such,
and by convention you should identify it with *'s.
Since X represents code that you will later execute then before making N
copies of it, you should compile it:
(defvar *l* nil)
(defun pump-l (n x)
(let ((code (compile nil `(lambda () ,x))))
(dotimes (i n) (push code *l*))))
> [2]> (defun eval-whole-L()(dolist (f L)(eval f)))
Instead of EVAL, use FUNCALL
(defun execute-whole-l ()
(dolist (f *l*)
(funcall f)))
> [3]> (pump-l 100000 '(progn (setq A ())(push 0 A)))
Don't generate code with undefined behaviours. a is local, use let, a is
global (defvar *a* nil)
> [4]> (time (eval-whole-l))
>
> Real time: 3.046875 sec.
> Run time: 3.046875 sec.
> Space: 91965272 Bytes
> GC: 144, GC time: 0.375 sec.
Instead of timeing the results of all your mistakes and drawing fantastical
conclusions from them you should try learning some of the basics of the
language. Since you do not even know how to create a local variable you
abviously need to start at the beginning.
> So, PUSH (and other macros) in general case - should not be
> used in the generated code. I think that now everone can agree
> about that.
It seems everyone agrees that you don't know what you are talking about.
That is my opinion as well.
Jock Cooper
Instead of replying to this with your handwave why don't you reply to
the articles that showed why you got the results above and why they don't
mean what you think they do. I'd like to see your reasoning, because after
following this thread for a few days it's clear to me that you are utterly
clueless.
Marcus Breiing
> I simulated situation in which these parts of code like
> '(progn (setq a ()) (push 0 a))
> are generated during runtime, 100 000 of them, and evaluated.
So it hurts when you use macros? Don't use them then.
An eval that _allows_ macros doesn't cost significantly over an eval
that doesn't. Nothing forces you to actually have macros in your code,
apart from macros in the CL package. The latter, however, would be
only a "quality of implementation" issue to you (just as
eval-via-compile would be, only less so.) It doesn't reflect on
whether CL-the-language per se can do the job.
Marcus
Coby Beck
> Right, that makes sense. Though, am I right in saying that you're
> cheating a little & expanding/compiling the closure only once, then
> calling it X times? Where Major is expanding/interpreting and
> executing the closure X times?
It is only cheating if the rules require doing it in the most ridiculous way
imaginable. If there is a real problem to solve then a good rule of thumb
is "Don't be an idiot." If you want to time the execution of idiotic code
then yes it is cheating to not be an idiot! ;)
As I said to him earlier, if you are actually in a situation where you must
generate the code to use it only once, even then the work must occur
somewhere whether in your own code generating the detailed implementation or
in the macro-expansion. Since he is hardcoding forms for his tests, he is
not testing anything interesting except what take more time macro-expanding
or doing nothing. Well surprise, doing nothing is faster.
Kaz Kylheku
Majorinc wrote:
> In article <1138755721.025782.194260
> @o13g2000cwo.googlegroups.com>, j_mck...@bigfoot.com says...
> > I have several methods and functions that are similar, and I have
> > re-written most of them to be generated by macros.
> >
> > I've heard the adage to only use macros when they are needed, but
> > technically, since only Lisp has this type of source-generating macro,
> > they are never *really* needed, because the same problem can be solved
> > in some other language without macros.
>
> From my point of view, macros are the worst thing that happened
> to Lisp, because they compromise the most important feature of
> Lisp, code=data, especially the ability of programs to generate
> and execute other programs dynamically.
>
> Imagine program that produces and execute million small and
> fast programs in a minute - it is much better that these small,
> generated programs contain only function calls. If these small,
> generated program contain macros, you'll have millions of macro
> expansions in a minute in RUNTIME, and it can turn otherwise
> feasible concept into nightmare ...
Buhahahahaha!
If you generate a program at run-time and pass it to the compiler, that
compiler or evaluator has to deal with the syntax of that program,
regardless of whether it contains user-defined operators or not. All
syntactic features are essentially macros, whether they are built into
the language interpreter or compiler or whether they are programmable.
IF, COND, OR, AND, LET. All of that is syntax that has to be dealt with
whether it's built in or macro material.
In order to dynamically compile data that represents code, it is
necessary not only to expand macros within it, but to then analyze it
syntactically, build some parse trees, analyze those, generate some
intermediate representation, optimize it, then generate
machine-specific code and optimize that as well. It's a CPU-consuming
process that can cons up a lot of garbage too.
The point is that macro expansion is just a "drop in the bucket"
compared to all the other processing that has to be done in order for
Lisp data to go in, and quality machine code to pop out!
So if you generate and compile a million functions a minute, you are
invoking this entire compiling process a million times a minute.
It's still a very useful tool to be able to do this, and can result in
nice performance improvements if you can leverage it properly.
One way to use this poorly would be to do a whole lot of compiling of
little programs in a tight inner loop!
In general, the proper way is to generate only a smal number of dynamic
functions, perhaps just one. You call the resulting function from a
loop. (Or compile the loop directly around it).
Why would you generate a million functions in a minute? Firstly, to
even begin to justify that, these functions would have to be different
from each other. You would have to have a million different cases
(maybe iterations in a loop) for which you are individually customizing
some parameter in the code that is being compiled, so that you are in
fact passing slightly different code to the compiler each time. There
is probably a better way to handle those varying parameters than by
varying the dynamically generated source code, and going through the
compiler!
Coby Beck
news:uek2k4...@agharta.de...
> On Fri, 3 Feb 2006 18:53:51 +0100, Majorinc, Kazimir <fa...@email.com>
> wrote:
>
>> In article <umzh84...@agharta.de>, spam...@agharta.de
>> says...
>>
>>> This is not an answer to you directly because your recent behaviour
>>> in this newsgroup disqualifies you from that.
>>
>> Edi, please, go home and fuck with your mother
>
> Yes, that's what I meant, thanks for confirming that. I'm wondering
> why others still insist on feeding you.
I only do it for lurkers, though I don't think it needs much more said.
bradb at least was hoping to learn something. I absolutely agree that
prolonged interaction with this fellow is pointless, but every now and then
the record needs to be set straight (as I am sure you agree).
Coby Beck
>I imagine Major was simply repeating the same form for simplicity.
> Imagine that you are generating X unique forms, then executing them.
> Assuming that the generation cost is the same, then you are better to
> generate pre-expanded code rather than macro expand, right?
Even that is not clear. Either do the work on your own or use the
implementation's macroexpander, but the work needs to be done.
Kenny Tilton
And when the troll responds to your lurker-intended correction with
another remark that needs setting straight?
Mind you, that is a good way to share with newbies. Take trolls as
launching ramps for good Lisp information. The trick is to brush off the
obnoxiousness and just pontificate happily about Lisp. Drives trolls nuts.
kenny
Kaz Kylheku
Majorinc wrote:
> In article <m6OdnYUZBP_doH_e...@speakeasy.net>,
> ja...@hpalace.com says...
>
> >
> > Okay, so Icon encourages you to use generators for your looping
> > construct. Can generators be implemented in Icon without generators?
> > Cause they can in Lisp, see the Series package.
> > <http://series.sourceforge.net/>.
>
>
>
> Hey, I do not claim that Icons control operations are *better*
> (neither they are worse) than Lisp's. It is way too extensive
> question. I Just mentioned other languages beside Lisp that I
> used and that had programmer-defined control operations ...
>
> But if you asked, yes, it is possible and relatively simple to
> define generators in the majority of the languages. Basically,
> one need to write functions so they "remember" what happened
> last time they are called "in the same context" and to continue
> where they stopped before ...
If you have to pass a context pointer into the function call then the
function is not a true generator.
What jared is asking is: if the feature known as "generators" were
removed from the Icon language, could you write an Icon program which
would put that feature in? The question isn't asking whether, as the
language user, you could kludge together instances of functions and
object that provide the behavior of generators, but whether you could
implement the language feature of generators: that exact feature which
was removed. Achieving that feature would mean that existing Icon
programs which rely on Icon generators would actually be able to run
unmodified with that implementation.
We can do this sort of thing in Lisp. Take away LOOP from a Lisp
implementation, and you can get portable code that implements it.
Existing code will happily work with it.
Take away the object system, CLOS, and it can be implemented by the
user. Some CLOS systems in use are in fact derivatives of a package
called PCL: Portable Common LOOPS (Lisp OO Programming System) once
written by Kiczales.
Majorinc
@o13g2000cwo.googlegroups.com>, kkyl...@gmail.com says...
>
> The point is that macro expansion is just a "drop in the bucket"
> compared to all the other processing that has to be done in order for
> Lisp data to go in, and quality machine code to pop out!
Well, you are partly right - but only partly. Although time
required for macro expansion can be quite low, in general case,
it is not bounded by linear (or in fact, any) function of the
size of code. All other processing you described is linearly
related to the size of code.
There are other reasons why macro expansion takes lot of time.
But, why to speculate - if we can look in horse mouths - in
example I posted, macro expansion of PUSH is responsible for
99% of total time ... and it can be even worse.
>
> So if you generate and compile a million functions a minute, you are
> invoking this entire compiling process a million times a minute.
Compiling is good only for parts of code that can be compiled
once and executed many times. In the case - compile once -
execute once - it is better to use interpreter.
>
> It's still a very useful tool to be able to do this, and can result in
> nice performance improvements if you can leverage it properly.
>
> One way to use this poorly would be to do a whole lot of compiling of
> little programs in a tight inner loop!
>
> In general, the proper way is to generate only a smal number of dynamic
> functions, perhaps just one. You call the resulting function from a
> loop. (Or compile the loop directly around it).
>
> Why would you generate a million functions in a minute? Firstly, to
> even begin to justify that, these functions would have to be different
> from each other.
It is natural for many problems in AI domain. Think about
theorem proving or satisfiability testing that might rely on
understanding of the logical formula as Lisp expressions,
genetic programming ...
Majorinc
jo...@mail.com says...
> Instead of replying to this with your handwave why don't you reply to
> the articles that showed why you got the results above and why they don't
> mean what you think they do.
Oh, I think I answered on literally all criticism. This is the
point: I try to prove that addition is slow, and I test it on
this way:
(time (dotimes (i 1000)(+ 100 100)))
People who criticize me say that
(time (progn
(setf (x (+ 100 100)))
(dotimes (i 1000) x)
)
)
proves that addition is fast.
> I'd like to see your reasoning, because after
> following this thread for a few days it's clear to me that you are utterly
> clueless.
From my point of view, it is clear that many people here (not
all) do not have a slightest clue what I actually did here -
including you, of course. Oh well, what can we do ...
Majorinc
cb...@mercury.bc.ca says...
> > So, PUSH (and other macros) in general case - should not be
> > used in the generated code. I think that now everone can agree
> > about that.
>
> It seems everyone agrees that you don't know what you are talking about.
> That is my opinion as well.
You wrote many notes, but they systematically miss my point and
focus on (for this purpose) irrelevant details. My opinion is
that you simply do not get it. Well, OK with me.
Majorinc
@g47g2000cwa.googlegroups.com>, kkyl...@gmail.com says...
> If you have to pass a context pointer into the function call then the
> function is not a true generator.
Of course not, but there are other ways to change context.
>
> What jared is asking is: if the feature known as "generators" were
> removed from the Icon language, could you write an Icon program which
> would put that feature in? The question isn't asking whether, as the
> language user, you could kludge together instances of functions and
> object that provide the behavior of generators, but whether you could
> implement the language feature of generators: that exact feature which
> was removed. Achieving that feature would mean that existing Icon
> programs which rely on Icon generators would actually be able to run
> unmodified with that implementation.
No, in that sense "suspend" (keyword generators use instead of
"return") is one of the primitives in Icon.
>
> We can do this sort of thing in Lisp. Take away LOOP from a Lisp
> implementation, and you can get portable code that implements it.
> Existing code will happily work with it.
>
> Take away the object system, CLOS, and it can be implemented by the
> user. Some CLOS systems in use are in fact derivatives of a package
> called PCL: Portable Common LOOPS (Lisp OO Programming System) once
> written by Kiczales.
Yes, these two are not primitive features of Lisp.
Where are we going?
Majorinc
cb...@mercury.bc.ca says...
> "bradb" <brad.be...@gmail.com> wrote in message
> news:1138990091.8...@f14g2000cwb.googlegroups.com...
> > Right, that makes sense. Though, am I right in saying that you're
> > cheating a little & expanding/compiling the closure only once, then
> > calling it X times? Where Major is expanding/interpreting and
> > executing the closure X times?
>
> It is only cheating if the rules require doing it in the most ridiculous way
> imaginable. If there is a real problem to solve then a good rule of thumb
> is "Don't be an idiot." If you want to time the execution of idiotic code
> then yes it is cheating to not be an idiot! ;)
I answered on that criticism many times, but it is here again,
so I'll do it once again: if I want to prove that addition is
slow, then I'll write this test:
(time (dotimes (i 1000)(+ 100 100)))
If someone tries to prove that addition is not slow by pointing
that this is "idiotic code," and that code should look like
(time (progn (setf x 200)(dotimes (i 1000) x))
I think he has very serious problems ...
>
> As I said to him earlier, if you are actually in a situation where you must
> generate the code to use it only once, even then the work must occur
> somewhere whether in your own code generating the detailed implementation or
> in the macro-expansion. Since he is hardcoding forms for his tests, he is
> not testing anything interesting except what take more time macro-expanding
> or doing nothing. Well surprise, doing nothing is faster.
Oh, no - I tested how much time is spent in macro-expanding vs
executing expanded code. And it turned macroexpanding ate 99%
of the processor time, executing 1%.
Geoffrey Summerhayes
<Majorinc>; "Kazimir" <fa...@email.com> wrote in message news:MPG.1e4f21999...@news.carnet.hr...
>
> You wrote many notes, but they systematically miss my point and
> focus on (for this purpose) irrelevant details. My opinion is
> that you simply do not get it. Well, OK with me.
Oh no, we get it, your argument is something like,
"When a Lisp program is created incompetently,
it runs inefficently, therefore Lisp is to
blame not the author of the program."
---
Geoff
Coby Beck
> In article <p8OEf.247276$OU5.33357@clgrps13>,
> cb...@mercury.bc.ca says...
>> "bradb" <brad.be...@gmail.com> wrote in message
>> news:1138990091.8...@f14g2000cwb.googlegroups.com...
>> > Right, that makes sense. Though, am I right in saying that you're
>> > cheating a little & expanding/compiling the closure only once, then
>> > calling it X times? Where Major is expanding/interpreting and
>> > executing the closure X times?
>>
>> It is only cheating if the rules require doing it in the most ridiculous
>> way
>> imaginable. If there is a real problem to solve then a good rule of
>> thumb
>> is "Don't be an idiot." If you want to time the execution of idiotic
>> code
>> then yes it is cheating to not be an idiot! ;)
>
> I answered on that criticism many times, but it is here again,
> so I'll do it once again: if I want to prove that addition is
> slow, then I'll write this test:
>
> (time (dotimes (i 1000)(+ 100 100)))
>
> If someone tries to prove that addition is not slow by pointing
> that this is "idiotic code," and that code should look like
>
> (time (progn (setf x 200)(dotimes (i 1000) x))
<sigh> Against my better judgement...
The above test, as does your macro test, does not say that addition is slow,
it only says addition takes time.
Your test is like:
(time (dotimes (i 1000) (+ 100 100)))
vs
(time (dotimes (i 1000) 10000))
Conclusion: addition is slow, it should be removed from the language and you
should just put the answer in instead.
likewise your test can be reduced to this:
(time (dotimes (i 1000) (macroexpand '(push a b))))
vs
(time (dotimes (i 1000) '(setq b (cons a b))))
Conclusion: macroexpansion is slow, it should be removed from the language
and you should just put the answer in directly.
Generating code results in: surprise! code. If it results in code that
contains macros then you simply have not done all the work, you have passed
the remaining work on to the implementation's macroexpander.
I can not say it any simpler. Good luck.
>> As I said to him earlier, if you are actually in a situation where you
>> must
>> generate the code to use it only once, even then the work must occur
>> somewhere whether in your own code generating the detailed implementation
>> or
>> in the macro-expansion. Since he is hardcoding forms for his tests, he
>> is
>> not testing anything interesting except what take more time
>> macro-expanding
>> or doing nothing. Well surprise, doing nothing is faster.
>
> Oh, no - I tested how much time is spent in macro-expanding vs
> executing expanded code. And it turned macroexpanding ate 99%
> of the processor time, executing 1%.
Right, how much time is spent macro expanding vs how much spent not macro
expanding...wow, macroexpanding takes time.
Please see above.
Kaz Kylheku
> In article <1138995027.317422.174060
> @o13g2000cwo.googlegroups.com>, kkyl...@gmail.com says...
>
> >
> > The point is that macro expansion is just a "drop in the bucket"
> > compared to all the other processing that has to be done in order for
> > Lisp data to go in, and quality machine code to pop out!
>
> Well, you are partly right - but only partly. Although time
> required for macro expansion can be quite low, in general case,
When you're optimizing, there is only so much you can do about the
pathological "general case", right?
Just because worst-case general cases can exist, that doesn't mean we
should shy away from arming ourselves with useful tools like macros.
> it is not bounded by linear (or in fact, any) function of the
> size of code.
Uh, the /execution/ of that code is also not bounded by any function of
the size of the code.
You keep saying that code with functions only is better than macros.
This is your reason?
> All other processing you described is linearly
> related to the size of code.
I only described the processing vaguely. I would not be so hasty to
jump to the conclusion that compiling code of size N is O(N). Code of
size N could give rise, within the compiler, to a graph structure of
size N, and some algorithms over graph structures of size N are
non-polynomial. Some types of optimizations require the searching of
large spaces.
What you seem to be saying is that you have a great deal of trust in
the compiler being well-written, as well as the macros that come from
the compiler vendor, but that you have less trust in the user-defined
macros. Why? Both of them are just code. Both of them are largely
written in Lisp. Both of them can be compiled (yes macros themselves
compile all the way to native code). Code that has a high computational
complexity can be put into a macro or into a compiler.
Which one are you more likely to have source code for? Which one is
more likely to be under your control? Your macros or the vendor's
macros and compiler?
If you don't trust user-defined macros, why do you trust other
user-defined code?
If you put together some code dynamically, and then you compile it and
run it, or eval it, just once, there is no telling where the most time
will be spent. Will it be spent in macro-expanding? Compiling after
macroexpansion? Or in actually running that code? Who knows?
So you think that because macroexpansion is unnecessary, you can
eliminate one of these --- without changing any other variables in the
situation!
> There are other reasons why macro expansion takes lot of time.
> But, why to speculate - if we can look in horse mouths - in
> example I posted, macro expansion of PUSH is responsible for
> 99% of total time ... and it can be even worse.
Your "non macro" version used SETQ which is also a macro! It is not
primitive.
Like SETF and PUSH, SETQ is a general-purpose assignment operator that
can assign a new value to any place. It's syntactically restricted to
symbols, but symbols can macro-expand to arbitrary places and SETQ has
to handle that.
E.g, in:
(macrolet (x (cdr y))
(setq x 42))
The (SETQ X 42) does the same job as (SETF (CDR Y) 42). The SETF
expander for the CDR function has to be retrieved and called.
In Lisp, there isn't any clearly delineated set of primitives. What is
primitive and what is not is purely an implementation choice. You can
remove just about anything from Lisp and implement it back as a macro.
The only constraint is that you don't remove so many things together
that what is left cannot implement them back. For instance if you
remove every I/O related function, you can't put back I/O since you
don't have any portable way to get into the operating system calls.
How about LET? You think that binding variables over a block of code is
primitive? If you don't have LET, you can use LAMBDA:
(let ((x 3) (y 4)) (+ x y))
can be rewritten as:
((lambda (x y) (+ x y)) 3 4)
A macro can do this rewriting job.
> > So if you generate and compile a million functions a minute, you are
> > invoking this entire compiling process a million times a minute.
>
> Compiling is good only for parts of code that can be compiled
> once and executed many times. In the case - compile once -
> execute once - it is better to use interpreter.
But which one? The general one built into the language (generate source
and feed it there) or something in your own program? How do you know
it's better? What if that piece of code that is run once contains big
loops and does a lot of processing? Is it always better to interpret
it?
Suppose I have a regular expression that I only want to use once to
find a single instance of a pattern inside some text. The choice isn't
just between EVAL and COMPILE. The choice is between just interpreting
the raw expression (the string), and analyzing that expression and
turning it into Lisp source code. Then the choice for that code is
between EVAL and COMPILE. So there are three main choices: don't
generate code, do generate code but use eval, and do generate code and
compile it.
Note that some Lisps like Corman don't have an interpreter. Everything
is compiled. EVAL compiles machine language, and so EVAL and COMPILE do
the same thing. The tradeoffs between EVAL and COMPILE are specific to
the Lisp implementation and maybe other factors.
> > Why would you generate a million functions in a minute? Firstly, to
> > even begin to justify that, these functions would have to be different
> > from each other.
>
> It is natural for many problems in AI domain. Think about
> theorem proving or satisfiability testing that might rely on
> understanding of the logical formula as Lisp expressions,
> genetic programming ...
If you had logical formulas represented as Lisp expressions, don't you
think you'd have control over the syntax of the operators that are
allowed in them? If they were implemented as macros, wouldn't you be
the one implementing them and making them as efficient as possible, if
that mattered? So couldn't you avoid the general cases where
macro-expansion runs with poorly bounded times?
Majorinc
>
> Right, how much time is spent macro expanding vs how much spent not macro
> expanding...wow, macroexpanding takes time.
Oh no, macroexpanding time vs. nothing time is certainly 100:0,
and it is really not important. The test shows that
macroexpanding vs executing time is 100:1 and it suggests that
macros are practically useless as building blocks for the
runtime generated code.
Hence, if one invests effort in the development of macros, that
very moment his problem matures to the point he needs runtime
generated code -- and it is quite a usual idea in AI field --
all these efforts will be lost.
Majorinc
sum...@NhOoStPmAaMil.com says...
>
> <Majorinc>; "Kazimir" <fa...@email.com>
> > You wrote many notes, but they systematically miss my point and
> > focus on (for this purpose) irrelevant details. My opinion is
> > that you simply do not get it. Well, OK with me.
>
> Oh no, we get it, your argument is something like,
> "When a Lisp program is created incompetently,
> it runs inefficently, therefore Lisp is to
> blame not the author of the program."
Hm ... it looks like bullshit to me.
Kaz Kylheku
> Here is the test for PUSH.
>
>
> [1]> (defun pump-l(n x)(setf L ())(dotimes(i n)(push x L)))
> [2]> (defun eval-whole-L()(dolist (f L)(eval f)))
>
> [3]> (pump-l 100000 '(progn (setq A ())(push 0 A)))
> [4]> (time (eval-whole-l))
>
> Real time: 3.046875 sec.
> Run time: 3.046875 sec.
> Space: 91965272 Bytes
> GC: 144, GC time: 0.375 sec.
> _________________________________________
>
> Now, look what happens without macroexpansion:
>
> _________________________________________
>
> [5]> (pump-l 100000 '(progn (setq A ())(setq A (cons 0 A))))
> [6]> (time (eval-whole-l))
SETQ is a macro. It's just a cheaper macro, on your implementation,
than PUSH.
So this code is not "without macroexpansion".
The SETQ macro is like SETF, with the constraint that the destination
operand must be, syntactically, a symbol. SETQ is not primitive with
respect to SETF.
SETQ and SETF are also not primitive with respect to PUSH. Consider
that PUSH should be able to update a place without scanning twice to
locate it.
For instance:
(push 42 (sixth some-list))
SOME-LIST is a list, and its sixth element is also a list. We are
pushing 42 onto that list. A naive expansion of PUSH would scan to the
sixth cons cell twice: once to fetch the list, and a second time to
assign there, i.e.
(setf (sixth some-list) (cons 42 (sixth some-list)))
But you see, PUSH won't do that, because the SIXTH function has a SETF
expander which allow smarter code than that to be generated. Both SETF
and PUSH use that expander, in different ways (and so do other macros
like DECF and INCF).
You can create your own user-defined SETF expanders for stuff like
this. For instance, suppose you write your own lookup data structure,
and you want a user to be able to update an entry without doing two
accesses using (INCF (MY-LOOKUP S) VAL).
Also, where do you think that your ability came from to replace PUSH by
SETQ? It's because the code is not really dynamic that you were able to
do that. The "dynamic" code is really a literal that is statically
embedded in the code.
The PUSH is clearly there in the source code where you can tweak
manually.
All you have done is macro-expand the PUSH into SETQ by hand. You don't
think that you could write Lisp code to do that?
Now suppose that each one of those 100,000 pieces of code were slightly
different. Where would you do the hand-optimization then? How would
you demonstrate the "without macros" version?
> Real time: 0.1875 sec.
> Run time: 0.1875 sec.
That is still an eternity, in terms of modern machine speeds. The code
has only done a hundred thousand CONS and assignment operations.
Why don't you time how long it takes to just do this. Be sure to
compile it too.
(defvar a ())
(defun test-it ()
(dotimes (n 100000) ((setf a nil) (push 0 a))))
(compile 'test-it)
(time (test-it))
That's the base line. Doing 100000 pushes onto a dynamic variable,
after clearing it each time.
The difference between the time and memory use of this and the dynamic
cases with eval is all overhead.
> Space: 800008 Bytes
> GC: 1, GC time: 0.015625 sec.
> _________________________________________
>
> Speed up of two orders of value (only 25% time is spent in
> (setq A (cons 0 A)) part.
You haven't profiled it down to how much time is actually spent in the
CONS function and in the actual assignment to A. You're still including
the overhead of EVAL and the macroexpansion of SETQ.
> So, PUSH (and other macros) in general case - should not be
> used in the generated code.
Other macros ... like SETQ?
> I think that now everone can agree
> about that.
What if LET is a macro? You can't make a move in Common Lisp without
using macros.
> Only thing that we might disagree is importance of that fact.
> From my point of view, it is huge, disasterous problem - the
> code generation is not something marginal for Lisp, it is its
> essential feature.
Lisp has no one essential feature.
Well, if you think that something else has better dynamic code
generation that's easier to optimize in situations when you generate
thousands of trivial code fragments, then use that!
That may be important to you, but it's not important to me.
Lots of useful projects out there don't do any run-time code
generation, or don't abuse it in the ways you have described
(generating a large number of completely trivial expressions that are
practically no-ops, and that are fed into eval just once).
> If macros should not be used *in generated
> code*, there is no good reason to invest any resources in their
> development.
Even if that were true, there could still be good reasons to invest
resources in the development of macros. It's a fallacy to assume that
one problem (real or not) could destroy the usefulness of a tool.
A chainsaw breaks on concrete or steel. That doesn't mean I will take
it away from the lumberjack who uses it on softwood.
Kaz Kylheku
> In article <1139075857.692096.256950
> @g47g2000cwa.googlegroups.com>, kkyl...@gmail.com says...
>
>
> > If you have to pass a context pointer into the function call then the
> > function is not a true generator.
>
> Of course not, but there are other ways to change context.
Like what? If the context isn't a parameter to the function, then it
has to be somewhere in the environment. Maybe associated with the
calling thread, or global.
> > What jared is asking is: if the feature known as "generators" were
> > removed from the Icon language, could you write an Icon program which
> > would put that feature in? The question isn't asking whether, as the
> > language user, you could kludge together instances of functions and
> > object that provide the behavior of generators, but whether you could
> > implement the language feature of generators: that exact feature which
> > was removed. Achieving that feature would mean that existing Icon
> > programs which rely on Icon generators would actually be able to run
> > unmodified with that implementation.
>
> No, in that sense "suspend" (keyword generators use instead of
> "return") is one of the primitives in Icon.
Right, so the answer is no. You need "suspend" and you can't write that
in Icon if it's taken away.
Why not? Is there some rationale document written by the Icon
developers which explains why suspend can't be written in Icon?
Or maybe they didn't think of it? Could it be that, like blind monkeys,
the instant they thought about developing a new language, they coughed
up a grammar and reached for their LALR parser generator?
I'm looking at the Icon sources and indeed they use Yacc. They don't
actually ship the grammar file. The cgram.g file is missing, and the
Makefile steps for processing cgram.g into cparse.c through yacc are
commented out.
So even if I wanted to hack at the syntactic level in Icon, I'd have to
ask them for the missing pieces.
By the way, Yacc actions are essentially macros. Some syntax is
matched, and the corresponding nodes are assigned to meta-variables
like $1, $2, ... These are macro variables essentially: variables bound
to pieces of the parsed source code. The action has to compute
something over these variables and return an expansion by assigning to
$$.
In Icon, apparently, the suspend syntax is handled by these (see
cgrammar.c):
#define Suspend0(x1,x2) $$ = tree5(N_Loop,x1,x1,x2,EmptyNode)
#define Suspend1(x1,x2,x3,x4) $$ = tree5(N_Loop,x1,x1,x2,x4)
In cparse.c, these are already macro-expanded in the tarball:
# line 386 "cgram.g"
{yyval = tree5(N_Loop,yypvt[-1],yypvt[-1],yypvt[-0],tree1(N_Empty) ) ;}
break;
You can see how yacc compiles the macro: The meta-variables become
references to the yacc stack, and the $$ result is just yylval.
It's not that much different from doing something like
(defmacro suspend (x1 x2 x3 x4)
(make-tree-node 'node-loop x1 x2 x3 x4))
:)
> Where are we going?
*whistle*
Marcin 'Qrczak' Kowalczyk
>> No, in that sense "suspend" (keyword generators use instead of
>> "return") is one of the primitives in Icon.
>
> Right, so the answer is no. You need "suspend" and you can't write that
> in Icon if it's taken away.
>
> Why not? Is there some rationale document written by the Icon
> developers which explains why suspend can't be written in Icon?
It seems that in Lisp you need THROW and you can't write that in
Common Lisp if it's taken away.
Is there some rationale document written by the Lisp developers which
explains why THROW can't be written in Common Lisp?
--
__("< Marcin Kowalczyk
\__/ qrc...@knm.org.pl
^^ http://qrnik.knm.org.pl/~qrczak/
Edi Weitz
> It seems that in Lisp you need THROW and you can't write that in
> Common Lisp if it's taken away.
You can.
<http://home.pipeline.com/~hbaker1/MetaCircular.html>
--
Lisp is not dead, it just smells funny.
Real email: (replace (subseq "spam...@agharta.de" 5) "edi")
Geoffrey Summerhayes
<Majorinc>; "Kazimir" <fa...@email.com> wrote in message news:MPG.1e5074853...@news.carnet.hr...
Agreed, it is bull, the blame lies entirely
with the programmer.
--
Geoff
Kaz Kylheku
> "Kaz Kylheku" <kkyl...@gmail.com> writes:
>
> >> No, in that sense "suspend" (keyword generators use instead of
> >> "return") is one of the primitives in Icon.
> >
> > Right, so the answer is no. You need "suspend" and you can't write that
> > in Icon if it's taken away.
> >
> > Why not? Is there some rationale document written by the Icon
> > developers which explains why suspend can't be written in Icon?
>
> It seems that in Lisp you need THROW and you can't write that in
> Common Lisp if it's taken away.
Sure I can. I just use something else, such as conditions or restarts.
In Icon, if the suspend syntax is removed, you can't even write a dummy
version that prints "suspend syntax invoked" and otherwise does
nothing. So we don't even have to get as far as semantics before we
encounter a problem.
Marcin 'Qrczak' Kowalczyk
>> > Why not? Is there some rationale document written by the Icon
>> > developers which explains why suspend can't be written in Icon?
>>
>> It seems that in Lisp you need THROW and you can't write that in
>> Common Lisp if it's taken away.
>
> Sure I can. I just use something else, such as conditions or restarts.
Oops, bad example. Make it LAMBDA instead.
I was primarily referring to asking for a document which would explain
an obvious thing: that some language features must be builtin such
that others can be built on top of them.
Yes, some languages don't allow to make custom constructs emulate the
syntax of builtin constructs. It's a disadvantage but not that serious
one. Semantics is more important. Consider adding coroutines or threads.
M Jared Finder
I realize that this doesn't change your point, but couldn't you define
lambda as long as you had defun and first class functions? I'm
imagining something like:
CL-USER> (defmacro lambda* (arg-list &body body)
(let ((name (gensym "LAMBDA")))
`(progn
(defun ,name ,arg-list ,@body)
(function ,name))))
CL-USER> (mapcar (lambda* (arg) (1+ arg))
'(1 2 3 4 5 6 7))
(2 3 4 5 6 7 8)
Of course, this is because defun has to use something just like lambda
underneath.
-- MJF
Kaz Kylheku
> "Kaz Kylheku" <kkyl...@gmail.com> writes:
>
> >> > Why not? Is there some rationale document written by the Icon
> >> > developers which explains why suspend can't be written in Icon?
> >>
> >> It seems that in Lisp you need THROW and you can't write that in
> >> Common Lisp if it's taken away.
> >
> > Sure I can. I just use something else, such as conditions or restarts.
>
> Oops, bad example. Make it LAMBDA instead.
Okay
(defmacro lambda (&whole form) `(function ,form))
Hahaha. Anything else? Maybe the FUNCTION operator, that would be a
good one.
> I was primarily referring to asking for a document which would explain
> an obvious thing: that some language features must be builtin such
That is the obvious thing.
> that others can be built on top of them.
This part, apparently, isn't.
What seems to be obvious that a builtin language feature must also have
a corresponding built-in read syntax which you hack into your
"grammar.y". These are two sense of the word "built in" which need not
be tied together.
Is that a conscious design decision, or just a knee-jerk reaction
drilled into one's head by a conventional computer science education?
Builtin feature -> grammar feature.
It's like some holy alliance that's not even questioned!
See if I made that decision somewhere, I /would/ document it. I would
say that the only target API for this language is the character syntax
because ...
> Yes, some languages don't allow to make custom constructs emulate the
> syntax of builtin constructs. It's a disadvantage but not that serious
> one.
In this newsgroup, that's a grave omission.
> Semantics is more important. Consider adding coroutines or threads.
But wouldn't it be a shame if these coroutines came with an API
consisting of a custom read syntax, right?
André Thieme
Kaz Kylheku schrieb:
> Marcin 'Qrczak' Kowalczyk wrote:
> > "Kaz Kylheku" <kkyl...@gmail.com> writes:
> >
> > >> > Why not? Is there some rationale document written by the Icon
> > >> > developers which explains why suspend can't be written in Icon?
> > >>
> > >> It seems that in Lisp you need THROW and you can't write that in
> > >> Common Lisp if it's taken away.
> > >
> > > Sure I can. I just use something else, such as conditions or restarts.
> >
> > Oops, bad example. Make it LAMBDA instead.
>
> Okay
>
> (defmacro lambda (&whole form) `(function ,form))
>
> Hahaha. Anything else? Maybe the FUNCTION operator, that would be a
> good one.
Okay, please implement QUOTE.
What if you take CAR away (and with it FIRST)?
Maybe EQ?
What abound COND (and at the same time IF, if that is implemented on
using COND)?
I think also CONS is hard to implement. One couldn't use LIST as that
is defined on using CONS...
André
--
Rob Warnock
+---------------
| E.g, in:
| (macrolet (x (cdr y))
| (setq x 42))
| The (SETQ X 42) does the same job as (SETF (CDR Y) 42).
Typo? ITYM this, yes?
(symbol-macrolet ((x (cdr y)))
(setq x 42))
-Rob
-----
Rob Warnock <rp...@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607
Eli Gottlieb
closures. Not pretty.
I've been looking into this whole sort of thing, and I've found that
most of CL can be built up from the special operators required in the
Hyperspec (maybe with some slight changes, like why is "if" primitive
instead of "cond"?), a few functions for accessing the dynamic
environment (like function, fdefinition, macro-function, symbol-value),
and a few functions for talking to the rest of the computer outside the
Lisp system.
Majorinc
kkyl...@gmail.com says...
> >
> > there are other ways to change context.
>
> Maybe associated with the calling thread, or global.
Yes, thats what I meant.
>
> Is there some rationale document written by the Icon
> developers which explains why suspend can't be written in Icon?
>
> Or maybe they didn't think of it? Could it be that, like blind monkeys,
> the instant they thought about developing a new language, ...
Equivalent of the "suspend" can be written in Icon easily, but not in
the "Icon without generators" which is not Icon any more. People get
their ideas on variety of ways. They dream snakes, Apples fall on their
heads ... I do not see this line of thinking going toward conclusion
...
>
> I'm looking at the Icon sources and indeed they use Yacc. They don't
> actually ship the grammar file. The cgram.g file is missing, and the
> Makefile steps for processing cgram.g into cparse.c through yacc are
> commented out.
>
> So even if I wanted to hack at the syntactic level in Icon, I'd have to
> ask them for the missing pieces.
It appears that you are strongly trying to prove that Lisp's macros are
better than Icon's PDCO. I do not say that you are wrong (or right),
but I think that you need to dig for differences harder.
Note that Lisp and Icon are different languages,
Joe Marshall
André Thieme wrote:
> Okay, please implement QUOTE.
See this comp.lang.scheme post by Oleg Kiselyov:
<2001051001...@adric.cs.nps.navy.mil>
Be careful what you ask for....
Jens Axel Søgaard
> Okay, please implement QUOTE.
<http://okmij.org/ftp/Scheme/quote-as-macro.txt>
--
Jens Axel Søgaard
Pascal Bourguignon
> Okay, please implement QUOTE.
Easy:
(defun eval (expression)
(cond ((symbolp expression) ...)
((atom expression) expression)
((eq 'quote (car expression))
(if (= 2 (length expression))
(cadr expression)
(error "QUOTE takes one argument: ~S" expression)))
(...)))
> What if you take CAR away (and with it FIRST)?
Easy:
(defun cons (car cdr) (lambda (sel) (if sel car cdr)))
(defun car (cons) (funcall cons t))
(defun cdr (cons) (funcall cons nil))
> Maybe EQ?
Easy:
(defun eq (a b)
(= (system:address-of a) (system:address-of b)))
> What abound COND (and at the same time IF, if that is implemented on
> using COND)?
Easy (I'll do IF, you'll implement COND as a macro):
(defun eval (expression)
(cond ((symbolp expression) ...)
((atom expression) expression)
((eq 'quote (car expression))
(if (= 2 (length expression))
(cadr expression)
(error "QUOTE takes one argument: ~S" expression)))
((eq 'if (car expression))
(if (<= 2 (length expression) 3)
(if (eval (cadr expression))
(eval (caddr expression))
(eval (caddrr expression)))
(error "IF takes two or three argument: ~S" expression)))
(...)))
> I think also CONS is hard to implement. One couldn't use LIST as that
> is defined on using CONS...
See above, CONS is trivial. All right, my implementation is
simplistic, you'd need to add some stuff for consp, null, nil, etc,
but it stays rather trivial.
Now, perhaps the important point is that for one class of things, you
can do staying at the same level, cons, car cdr, while for other
things you need to go up one metalinguistic level. (See SICP chapter 4).
That's how you distinguish special operators from normal functions and
macros.
You may believe that I cheated for EQ. But it's easy to add a unique
identifier to all functions^W objects:
(defconstant t (lambda (then else address type) then))
(defconstant nil (lambda (then else address type) else))
(defconstant address (lambda (then else address type) address))
(defconstant type (lambda (then else address type) type))
(defun cons (car cdr)
(incf *address*)
(lambda (sel) (funcall sel car cdr *address* +cons-type+)))
(defun make-string (length &key initial-element)
(incf *address*)
(lambda (sel) ... *address* ...))
(defun car (cons) (funcall cons t))
(defun cdr (cons) (funcall cons nil))
(defun address-of (object) (funcall object address))
(defun type-of (object) (funcall object type))
--
__Pascal Bourguignon__ http://www.informatimago.com/
You're always typing.
Well, let's see you ignore my
sitting on your hands.
Eli Gottlieb
> "André Thieme" <address.good.un...@justmail.de> writes:
>
>
>>Okay, please implement QUOTE.
>
>
> Easy:
>
> (defun eval (expression)
> (cond ((symbolp expression) ...)
> ((atom expression) expression)
> ((eq 'quote (car expression))
> (if (= 2 (length expression))
> (cadr expression)
> (error "QUOTE takes one argument: ~S" expression)))
> (...)))
>
>
>
>>What if you take CAR away (and with it FIRST)?
>
>
> Easy:
>
> (defun cons (car cdr) (lambda (sel) (if sel car cdr)))
> (defun car (cons) (funcall cons t))
> (defun cdr (cons) (funcall cons nil))
I don't think that's allowed. They work, but they're not primitive
implementations of cons, car, and cdr because they all require at least
cons to work. In order for the Lisp system to evaluate a defun or a
lambda in the first place, it must know how to read the text into Lisp
objects. For lists (like lambdas) this involves consing. Therefore
you're definining cons by having the Lisp system do implicit consing.
Pascal Bourguignon
This is a question of bootstrapping. You only need to re-evaluate the
(defun read ...) to let it use the new cons/car/cdr, and then you can
reread the forms:
(defun cons (car cdr) (lambda (sel) (if sel car cdr)))
(defun car (cons) (funcall cons t))
(defun cdr (cons) (funcall cons nil))
which will be read with the new cons implementation.
Have a look at the build process of any implementation of Lisp, or any
compiler written in itself, including gcc, it involves such
bootstraping phases.
--
__Pascal Bourguignon__ http://www.informatimago.com/
ATTENTION: Despite any other listing of product contents found
herein, the consumer is advised that, in actuality, this product
consists of 99.9999999999% empty space.
Thomas A. Russ
> Oh no, macroexpanding time vs. nothing time is certainly 100:0,
> and it is really not important. The test shows that
> macroexpanding vs executing time is 100:1 and it suggests that
> macros are practically useless as building blocks for the
> runtime generated code.
This is not the correct conclusion, as has been pointed out by several
people (including myself) before. Whether they are useful or useless
depends on how much time using the macros saves in the code that is
doing the code generation. If you have a macro that provides a useful
abstraction for your code generation, then you can either do the
equivalent of the macro expansion in your code generation procedure or
else you can have the compiler do it.
But in your examples, you are only measuring the cost of having the
compiler do it for you. You are NOT measuring the cost of having to do
the equivalent of the macroexpansion in your code generator.
> Hence, if one invests effort in the development of macros, that
> very moment his problem matures to the point he needs runtime
> generated code -- and it is quite a usual idea in AI field --
> all these efforts will be lost.
What a curious idea. I've been involved in the AI field for over 25
years and haven't found this to be a problem. In the Loom knowledge
representation system, extensive use is made of macros. In fact the
entire query system is implemented as one gigantic macro. The runtime
version of this involves calling the compiler. That is mainly because
the expansion of the macros that are used to describe the query create
loops and it would be totally idiotic to try to run loops interpreted.
Loom does not have any runtime problems that can be traced to the
process of macroexpansion.
That is generally because, when using lisp, it is rare that one actually
has to generate code at run-time and then execute it only once. Most of
the macro-expanded code in Loom involves things like relation
definitions which are potentially executed many, many times. It is in
that expansion that terrific use is made of the run-time access to the
Lisp compiler, so that the code did not require us to build a run-time
interpreter of our own. We compile the code down to lisp and then store
a compiled procedure for doing the inference. Now, that is FAST!
Link for Loom: http://www.isi.edu/isd/LOOM/
About the only application that I can see where you might be generating
lots and lots of single-use runtime code would be in some form of
genetic algorithm. And that is certainly not the whole of AI.
There is also a nice regular expression package nregex.cl (available at
the CMU archives among other places) that does a similar sort of thing
for regular expressions. It builds a lisp procedure for testing the
regular expression and then compiles it. That produces a fast,
specialized procedure for recognizing the particular regular
expression. This is generally only a real win if you want to use that
particular regular expression many times -- such as applying it in a
search across a large number of objects -- but otherwise the time isn't
much of a problem.
Now back to the original problem. I would hope that any loops in the
code that the OP generates is done in terms of tagbody and go. Because
any other loops will also be macros. But if you're running any code
with significant loops in it, evaluating without compiling is a really
stupid thing to be doing.
--
Thomas A. Russ, USC/Information Sciences Institute
Thomas A. Russ
> Only thing that we might disagree is importance of that fact.
> From my point of view, it is huge, disasterous problem - the
> code generation is not something marginal for Lisp, it is its
> development.
Well, this might be the case for your particular, specialized
application. Even then there is some dispute, because you will have to
do your own macro-expansion of any non-trivial macros. In that sense
the PUSH versus its expansion is almost too trivial to deal with. But
in any case, it is precisely those areas where there is a performance
problem for your code that need to be addressed with lower-level
implementations.
For the bulk of Common Lisp programming, repeated run-time code
generation is an irrelevant concern. Just as for most programming in
any other language. Most code generation in Lisp is not done at run
time but at compile time. In that case, macro creation is a great thing
to be doing. It is often the only way to introduce useful abstractions
into the code and provide the abstractions that are necessary.
I suppose that if you run the appropriate tests, you will discover that
there is also an overhead to function calls. Based on that, I would
then logically expect you to call for the removal of functions from lisp
as well, since if you put everything into a single procedure without
function calls, you can avoid the overhead of the function calls as
well.
The useful lesson from this excessively verbose debate is perhaps this:
For whatever application you are building, if there is a performance
problem it is worthwhile testing carefully to figure out where it comes
from and then trying to ameliorate the effects.
The caution is that any such analysis is really limited to the
circumstances that hold for that particular application, and can only be
broadly generalized if you are able to take a wider view. Making
blanket statements based on a very narrow application domain is
counter-productive.
(And it is clearly what has sparked this voluminous discussion)