LISP apply question
jennyjenny
***Background - skip if you want
I am working on several high level math and statistics classes for
PHP. Right now I am working on a binomial linear regression
implementation. As background, I am studying implementations in other
languages. One which I found source code for the algorithm is in LISP.
In order to understand the algorithm I need to decipher certain
elements of the code. So, I have spent the last two days reading code
samples and an assortment of online resources. Online I have read
large parts (of the applicable sections) of Successful LISP, the
Common LISP HyperSpec, and several other sites as well.
I program fluently in PHP and Java for the web and C/C++ offline. I
have no LISP experience prior to 3 days ago.
***End of background stuff
Right now I have basically one question, well to be honest not knowing
LISP very well it may be multiple questions.
Here is the line of code I'm trying to translate into unLISPed
english.
**start of code chunk
(defun binomialreg-model (x y n &rest args)
"Args: (x y n &rest args)
Returns a binomial regression model. Accepts :LINK, :OFFSET and
:VERBOSE
keywords in addition to the keywords accepted by regression-model."
(apply #'send binomialreg-proto :new :x x :y y :trials n args))
**end of code chunk
Here's what I know:
This is a Constructor function named binomialreg-model that accepts as
arguements 3 required parameters, x, y, and n and that potentially
accepts an unspecified number of additonal paraments, listed as args.
The function will return the object so created.
This is a object is an instance of a prototype called
binomialreg-proto.
Here's what I think:
To constuct the object, the function uses 1 line - the (apply .....)
line.
That line references a function send. That function somehow is applied
to the trailing args.
The reference binomialreg-proto refers to the prototype for the
binomialreg-model model.
Here's where I'm lost after reading the apply examples I've found:
Basically, I'm not sure what its doing.
Is it doing (send binomialreg-proto :new <args>) with all the things
after :new as <args>?
Is it doing some sort of sequence like:
(send binomialreg-proto :new)
(send binomialreg-proto :x)
(send binomialreg-proto x)
and so on...
Is it doing the above series all at once?
Is it doing something completely different?
This whole apply thing is thoroughly confusing me and unfortunately
for me the online examples make little sense. I mean really, the
examples start with
(setq f '+) => +
(apply f '(1 2)) => 3
which seems unusually silly to me. Why not just (+ 1 2)?
Then, it dives all the way to the deep end with
(defun foo (size &rest keys &key double &allow-other-keys)
(let ((v (apply #'make-array size :allow-other-keys t keys)))
(if double (concatenate (type-of v) v v) v)))
(foo 4 :initial-contents '(a b c d) :double t)
=> #(A B C D A B C D)
which has 50 things going on at once and no clear point as to what is
the apply doing.
Any help on explaining this in nonLISP terms would be greatly
appreciated.
Jenny
Pascal Costanza
jennyjenny wrote:
[...]
> Is it doing (send binomialreg-proto :new <args>) with all the things
> after :new as <args>?
> Is it doing some sort of sequence like:
> (send binomialreg-proto :new)
> (send binomialreg-proto :x)
> (send binomialreg-proto x)
> and so on...
> Is it doing the above series all at once?
> Is it doing something completely different?
The starting point for understanding how Common Lisp behaves in this
regard can be found in the HyperSpec in 3.1.2.1 Form Evalution,
especially 3.1.2.1.2 Conses as Forms.
What is meant by "cons" here is what you usually type in as a list in a
program source. So that section describes what happens when you type in
either a macro application, a special form, a function application or a
lambda form. It's a good idea to first try to understand function
application and then to proceed to other cases.
When you see a function application, what happens in the background is
that all parameters to a function are first evaluated from left to
right, results are pushed to an argument stack and then the function is
called. This is what always happens, no matter how the arguments are
processed later on.
So when you see something like (send a b c :d e), the five arguments are
first evaluated, the results are pushed, and then send is called. In
this example, :d is a symbol from the so-called keywords package. All
symbols from that package evaluate to themselves. At the call site,
nothing special happens with such keyword symbols. It's really the case
that the symbol :d is pushed as the fourth argument to send.
In a function declaration, a programmer can use lambda list keywords to
declare special processing of the arguments. By default, one only
declares mandatory arguments.
(defun f (x y) (+ x y))
...takes two mandatory arguments, nothing more, nothing less.
The following lambda list keywords are the most widely used ones:
&optional, &rest, &key and &allow-other-keys.
(defun g (x &optional y) (list x y))
...takes one mandatory argument and one optional argument. You can call
(g some-arg) or (g some-arg1 some-arg2).
(defun h (x &rest args) (list x args))
...takes one mandatory argument and an arbitrary number of additional
arguments. Those additional arguments are stored in a list referred to
by args.
(defun i (&key x &key y) (list x y))
...takes two keyword arguments. You can call (i :x 5 :y 7) or (i :y 4 :x
8) or some such. As I have said above, these calls push four arguments
to the stack. The Common Lisp run-time system takes care of processing
the arguments correctly so that you see the right values bound by the
right variables in a function body.
Keyword arguments are usually checked at run time. So you can't call (i
:z 7) or some such. If you want a function to be more liberal you can
use the &allow-other-keys lambda list keyword. In order to be able to
access such additional keyword arguments, you need to combine this with
a &rest declaration. The exact details are given in the HyperSpec. See
3.4 Lambda Lists.
There are two ways in Common Lisp to call a function that is stored in a
variable.
(funcall f a b c)
...evaluates the arguments a b c, pushes the results to the argument
stack and then calls the function that is stored in f. You could refer
to a function directly like this:
(funcall #'h a b c)
This calls the above defined function h. However, this is equivalent to
calling it directly:
(h a b c)
apply is similar to funcall. However, it takes a list as the last
argument and uses all the values stored in that list as separate
arguments. So (h a b c) could be expressed thus:
(apply #'h (list a b c))
(apply #'h a (list b c))
...and so on. This is especially useful when dealing with &rest
arguments. See the following function.
(defun j (&rest args) (apply #'+ args))
This computes the sum of an arbitrary number of arguments. The advantage
of apply is that you don't need to splice up the arguments by hand.
There are further advantages. For example, especially cool is the fact
that (apply f :k 5 a-list) overrides any value that is passed for :k in
a-list with 5. This is because Common Lisp is very consistent with its
left-to-right evaluation rule. But this might be too advanced at the
moment. This is mainly to say that some things in Common Lisp that look
strange in the beginning make a lot of sense in more complex scenarios.
However, it's hard to describe them all without writing a book. ;)
I hope this is a good first step to understand what's happening. Now
comes the most important advice: You should install yourself a Common
Lisp system and try out these or other examples you find in tutorials by
yourself. These things are much easier to understand when you actually
see what they do. They look much harder than they are in practice.
I know you only want to understand other people's Lisp code. But I think
it's really easier that way. It's very likely that the investment will
pay off.
Also make sure that you use an editor that actively supports Lisp
syntax. Among the best choices are emacs or the editors integrated into
commercial Common Lisp implementations. Look out for trial versions
which are generally pretty good. See http://alu.cliki.net/Implementation
for a list of available Common Lisp implementations.
All the best,
Pascal
--
Tyler: "How's that working out for you?"
Jack: "Great."
Tyler: "Keep it up, then."
Erik Naggum
| Here is the line of code I'm trying to translate into unLISPed
| english.
|
| **start of code chunk
| (defun binomialreg-model (x y n &rest args)
| "Args: (x y n &rest args)
| Returns a binomial regression model. Accepts :LINK, :OFFSET and
| :VERBOSE
| keywords in addition to the keywords accepted by regression-model."
| (apply #'send binomialreg-proto :new :x x :y y :trials n args))
| **end of code chunk
| Is it doing (send binomialreg-proto :new <args>) with all the things
| after :new as <args>?
Yes, except that the rest argument list called ARGS above is spread
into its constituent elements first.
(apply #'+ 1 2 '(4 5 6)) is just like (+ 1 2 3 4 5 6), but if you have
a list of arguments, this is the most obvious way to spread them out.
There is in fact no way to call a function /directly/ with a list of
arguments. This is in sharp contrast to, e.g., C, where using varargs
places significant restrictions on how you can use the argument list.
| Any help on explaining this in nonLISP terms would be greatly
| appreciated.
Lisps have traditionally regarded the arguments as a list, and even
though modern Lisp systems use registers and the stack like everybody
else, conceptually, functions are passed an argument list and hack it
up into its individual pieces. For instance, keyword arguments, like
:X X :Y Y :TRIALS N above, are passed in a list which is parsed by the
recipient. Unlike Ada and Perl and other languages that allow callers
to name arguments, but still pass them in a predefined order, Common
Lisp conceptually if not actually defers the parsing of the argument
list to the function that receives it.
The closest thing to this model is the Unix argument vector which it
is common to refer to using $* for the entire vector, and $1, $2,
... for the individual arguments. Unix programs have no support at
all for handling the argument vector, so every program needs to hack
it up into relevant pieces, picking up any options and any required
arguments before usually treating the final arguments as a list of
files to work on.
In other words, a call to BINOMIALREG-MODEL takes three or more
arguments, does a slight massage on the argument list and passes it
onwards to SEND BINOMIALREG-PROTO :NEW, which accepts the three
keyword arguments X, Y, and TRIALS. Any excess arguments to the first
function therefore have to be keyword arguments to the latter. To see
what a call to BINOMIALREG-MODEL does, the excess arguments only have
meaning when you look at what SEND BINOMIALREG-PROTO :NEW accepts. My
guess is that you will find a call to BINOMIALREG-MODEL with more than
three arguments extremely rarely, because they would have the form
(binomialreg-model x y n :foo foo :bar bar). If you never find any
calls like that, you should ignore the flexibility and complexity of
APPLY and consider (binomialreg-model x y n) /exactly/ the same as a
call to (send binomialreg-proto :new :x x :y y :trials n).
--
Erik Naggum | Oslo, Norway
Act from reason, and failure makes you rethink and study harder.
Act from faith, and failure makes you blame someone and push harder.
Kenny Tilton
jennyjenny wrote:
> This whole apply thing is thoroughly confusing me and unfortunately
> for me the online examples make little sense. I mean really, the
> examples start with
> (setq f '+) => +
> (apply f '(1 2)) => 3
> which seems unusually silly to me. Why not just (+ 1 2)?
You are right, but don't be too hard on the tech writer, it is hard
conveying concepts in simple examples that could not be done infinitely
easier some other way. How about this simple change:
(setq addends (list 1 2 3))
(apply '+ addends)
The point is (+ addends) will not work. ie, what happens when in the
course of your algorithm you end up with a list of things and you have
an operator which expects not a list argument (you can't do "(+ '(1 2
3))" but instead a list /of/ arguments.
In the case of copy-list, you could do:
(copy-list addends)
and (funcall 'copy-list addends), or more realistically:
(funcall some-list-expecting-fn addends)
But '+ does not work that way, so funcall won't work, you need apply.
kenny
--
http://tilton-technology.com
Why Lisp? http://alu.cliki.net/RtL%20Highlight%20Film
Your Project Here! http://alu.cliki.net/Industry%20Application
Marco Antoniotti
apart from th eexcellent comments people have already made, there is one
I feel obliged to add...
jennyjenny wrote:
> Hello, complete LISP newbie here.
>
> **start of code chunk
> (defun binomialreg-model (x y n &rest args)
> "Args: (x y n &rest args)
> Returns a binomial regression model. Accepts :LINK, :OFFSET and
> :VERBOSE
> keywords in addition to the keywords accepted by regression-model."
> (apply #'send binomialreg-proto :new :x x :y y :trials n args))
> **end of code chunk
>
> Here's what I know:
> This is a Constructor function named binomialreg-model that accepts as
> arguements 3 required parameters, x, y, and n and that potentially
> accepts an unspecified number of additonal paraments, listed as args.
> The function will return the object so created.
>
The above code uses a "send" function? Do you know that in Common Lisp
you can do
(defun binomial-model (x y n &rest args)
(apply #'new binomial-proto :x x :y y :trials n args))
;; Assuming ARGS is a keyword/value pairs list.
I.e. You can define your NEW function to do exactly what you need.
There is not need to write whole SEND-based infrastructure.
....
> Is it doing (send binomialreg-proto :new <args>) with all the things
> after :new as <args>?
> Is it doing some sort of sequence like:
> (send binomialreg-proto :new)
> (send binomialreg-proto :x)
> (send binomialreg-proto x)
> and so on...
> Is it doing the above series all at once?
> Is it doing something completely different?
Your code depends on the definition of SEND. SEND is not a Common Lisp
primitive, therefore I do not know what it does. I just guess what it
does :)
My suggestion is that you read up a little bit more about CLOS before
you continue. It will be time very well spent.
Cheers
--
Marco
Kaz Kylheku
> (apply f '(1 2)) => 3
> which seems unusually silly to me. Why not just (+ 1 2)?
Because the expression (+ 1 2) fixes a known function and known
argument list in the source code of the program itself. The expression
(+ 1 2), when evaluated, can never do anything other than call the +
function with the arguments 1 2.
Your question is like, why do we ever have
(print user-name)
instead of
(print "Joe User")
what's the problem? Just make sure you ship the program only to people
called "Joe User".
What apply does is take some list of unknown length and turn its
elements into the arguments of a function call, dynamically---that is
to say, at run time, just before the call happens. Moreover, the
identity of the function is itself the result of an expression.
Apply is often, though not always or necessarily, used together with
trailing argument lists captured by &rest. That is to say, the need to
use apply arises when a function with a variable-length argument list
needs to call another such function, and pass down the trailing
arguments which it was given, and which were captured as a list and
bound to a single variable specified by &rest.
> Then, it dives all the way to the deep end with
> (defun foo (size &rest keys &key double &allow-other-keys)
> (let ((v (apply #'make-array size :allow-other-keys t keys)))
> (if double (concatenate (type-of v) v v) v)))
> (foo 4 :initial-contents '(a b c d) :double t)
> => #(A B C D A B C D)
> which has 50 things going on at once and no clear point as to what is
> the apply doing.
FOO is a variadic function; it takes a size parameter followed by a
zero or more arguments which end up as a list bound to the KEYS
variable---that is what &REST does. MAKE-ARRAY is another variadic
function. So when FOO wants to call MAKE-ARRAY and pass along the
trailing arguments, APPLY must be used. So this is exactly that use
that I mentioned; i.e APPLY in conjunction with &REST.
So effectively, FOO is a wrapper for MAKE-ARRAY which somehow
simplifies it for some intended use.