keywordp, load-time-value
Tamas K Papp
I was reading the code of (metabang-)bind, and came across a code
snippet for detecting if something is a keyword. Extracted from
context (the original is in bind.lisp, in bind-macro-helper) and
modified, it looks like
(defun keywordp2 (symbol)
(and (symbolp symbol)
(eq (symbol-package symbol)
(load-time-value (find-package :keyword)))))
Questions:
1. How is this different from what keywordp does? (the implicit
assumption is that it is likely to be different, because otherwise this
code would just use keywordp, but I could be wrong).
2. Why is load-time-value needed? (it is still one of the more obscure
parts of CL for me).
Thanks,
Tamas
Pascal J. Bourguignon
IIUC, it is not necessary, but used to cache the package, to avoid
evaluating (find-package :keyword) everytime.
--
__Pascal Bourguignon__
Tamas K Papp
So is keywordp in CL semantically equivalent to
(defun keywordp3 (symbol)
(and (symbolp symbol)
(eq (symbol-package symbol)
(find-package :keyword))))
? By the way, you are right about speed:
(time ; SBCL's built-in keywordp
(dotimes (i 100000000)
(keywordp :foo)))
(time ; with load-time-value
(dotimes (i 100000000)
(keywordp2 :foo)))
(time ; without load-time-value
(dotimes (i 100000000)
(keywordp3 :foo)))
gives
Evaluation took:
0.040 seconds of real time
0.040003 seconds of total run time (0.040003 user, 0.000000 system)
100.00% CPU
101,419,625 processor cycles
0 bytes consed
Evaluation took:
0.649 seconds of real time
0.644040 seconds of total run time (0.644040 user, 0.000000 system)
99.23% CPU
1,639,612,524 processor cycles
66,584 bytes consed
Evaluation took:
22.094 seconds of real time
21.837364 seconds of total run time (21.817363 user, 0.020001 system)
[ Run times consist of 0.308 seconds GC time, and 21.530 seconds non-GC
time. ]
98.84% CPU
55,829,693,527 processor cycles
801,813,224 bytes consed
but of course it should not matter as bind is a macro, and it is not
likely to be called 100 million times in most programs.
Tamas
Rob Warnock
+---------------...
| >> 2. Why is load-time-value needed? (it is still one of the more obscure
| >> parts of CL for me).
| >
| > IIUC, it is not necessary, but used to cache the package, to avoid
| > evaluating (find-package :keyword) everytime.
|
| So is keywordp in CL semantically equivalent to
|
| (defun keywordp3 (symbol)
| (and (symbolp symbol)
| (eq (symbol-package symbol)
| (find-package :keyword))))
| ?
Yeah, basically. CMUCL defines it this way [file "src/code/symbol.lisp"]:
(declaim (special *keyword-package*))
(defun keywordp (object)
"Returns true if Object is a symbol in the keyword package."
(and (symbolp object)
(eq (symbol-package object) *keyword-package*)))
where EXTENSIONS:*KEYWORD-PACKAGE* is initialized to the keyword
package during startup... which is essentially equivalent to the
LOAD-TIME-VALUE in your original version.
Six of one, half dozen of the other...
-Rob
-----
Rob Warnock <rp...@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607
Willem Broekema
> (defun keywordp2 (symbol)
> (and (symbolp symbol)
> (eq (symbol-package symbol)
> (load-time-value (find-package :keyword)))))
> 1. How is this different from what keywordp does?
It's not.
> 2. Why is load-time-value needed? (it is still one of the more obscure
> parts of CL for me).
Here it's not needed for semantics, but it's an efficiency win: the
lookup of the package by name now only happens once (at fasl loading
time), instead of at every function call.
Functions that internally use some data structure that does not
change, make good candidates for load-time-value. Here's an example
from CLPython. The function checks whether a character should be
classified as punctuation:
(defun punct-char1-p (c)
(declare (optimize speed))
(let ((code (char-code c)))
(let ((arr (load-time-value
(loop with arr = (make-array 128 :element-type
'bit :initial-element 0)
for ch across "`=[]()<>{}.,:|^&%+-*/~;@"
do (setf (sbit arr (char-code ch)) 1)
finally (return arr)))))
(and (< code 128)
(= (sbit arr code) 1)))))
The bitvector is created only once, so the check is very fast. I find
it very elegant that l-t-v forms (like #.) can appear anywhere in an
expression, like as a let value here. For debugging, you could simple
unwrap the load-time-value and the loop form would be exactly where it
should be.
But things can go wrong when using load-time-value to reference
packages, classes or special variables. The order in which load-time-
value forms are evaluated during the loading of a file is explicitly
unspecified. (It could be that when loading a fasl, first all l-t-v
forms are evaluated and only after that the top-level forms. CMUCL
seemed to do something like that.) So it's *not* portable to use load-
time-value to refer to e.g. package or class definitions earlier in
the same file. I was initially surprised by this, but it makes sense
to give compiler writers some freedom here. The following examples are
all wrong:
(defpackage p ..)
..
(defun foo ()
(.. (load-time-value (find-package :p)) ..)
or:
(defclass c ..)
..
(defun foo ()
(.. (load-time-value (find-class 'c)) ..)
or:
(defvar *bar* ..)
..
(defun foo ()
(.. (load-time-value *bar*) ..))
To fix these cases, move the defining form to a file that's guaranteed
to be loaded earlier than the file that does l-t-v. (Using eval-when
around the defining form does not solve the problem.)
I thought load-time-value could also be used to keep state in a
function, but it turns out that is not reliable at all: in interpreted
mode the l-t-v form is evaluated several times:
cl-user(20): (defun foo ()
(car (load-time-value (progn (warn "l-t-v eval!") (list
1 2 3)))))
foo
cl-user(21): (foo)
Warning: l-t-v eval!
1
cl-user(22): (foo)
Warning: l-t-v eval!
1
Only when the function is compiled, is the evaluation happening once
and for all:
cl-user(23): (compile 'foo)
; While compiling foo:
Warning: l-t-v eval!
foo
t
t
cl-user(24): (foo)
1
cl-user(25): (foo)
1
For the function keywordp2, it's also possible to use read-time
evaluation instead of load-time evaluation, by using "#.". I actually
prefer this, as it's a shorter notation:
(eq (symbol-package symbol) #.(find-package :keyword))
Read-time evaluation only works when the resulting object (here the
package) can be externalized, see CLHS 3.2.4 "Literal Objects in
Compiled Files". This is true for e.g. numbers and arrays, but not
always for CLOS instances (see make-load-form). For package it's true
as long as you guarantee that a package with that name exists when the
file is loaded.
Another nice use case for #. is in functions doing interning. They
should ensure the interning happens in the intended package, not in
whichever package is current at the time of function call:
(defun my-interning-function (foo bar)
(.. (intern (format nil "~A.~A" foo bar) #.*package*) ..))
where #.*package* means: the value of *package* (the "current
package") at the time the defun expression is read. Thus symbols are
orderly interned in the same package as where the function lives.
Yet another good use for #. is when using macros that don't evaluate
their arguments, but you really want to put something "variable" in
them, like:
(defconstant +foo+ 1)
(defconstant +bar+ 2)
;; assuming the constants' values are available at this point:
(defun classify (x)
(case x
(#.+foo+ (warn "it's a foo!"))
(#.+bar+ (warn "it's a bar!"))))
Or declarations:
(defconstant +decl-optimize-debug+ '(declare (optimize (safety 3)
(debug 3))))
(defun foo ()
#.+decl-optimize-debug+
..)
(defun bar ()
#.+decl-optimize-debug+
..)
Or in this system definition file, where the dependency on :yacc is
optional for Allegro (only include it as depency if it is available),
but required in other implementations. It makes for a nice mess of #+,
#- and #. :)
(asdf:defsystem :clpython.parser
:description "Python parser, code walker, and pretty printer"
:depends-on
#-allegro (:clpython.package :yacc)
#+allegro #.`(:clpython.package ,@(when (asdf:find-system :yacc
nil) `(:yacc))))
- Willem