I now have learnt about arrays and aref in Lisp. So far, it's quite easy to grasp, and it works like a charme:
(defparameter *foo* (make-array 5))
(aref *foo* 0) ; => nil
(setf (aref *foo* 0) 23)
(aref *foo* 0) ; => 23
What puzzles me is the aref "magic" that happens when you combine aref and setf. It seems as if aref knew about its calling context, and would then decide whether to return a value or a place that can be used by setf.
Anyway, for the moment I just take this as granted, and don't think about the way this works internally too much.
But now I wanted to create a function that sets an element of the *foo* array to a predefined value, but I don't want to hardcode the *foo* array, instead I want to hand over a place:
(defun set-23 (place)
…)
So basically this function sets place to 23, whatever place is. My initial naive approach was
(defun set-23 (place)
(setf place 23))
and call it using:
(set-23 (aref *foo* 0))
This does not result in an error, but it also doesn't change *foo* at all. My guess would be that the call to aref resolves to nil (as the array is currently empty), so this would mean that
(setf nil 23)
is run, but when I try this manually in the REPL, I get an error telling me that:
NIL is a constant, may not be used as a variable
(And this absolutely makes sense!)
So, finally I have two questions:
What happens in my sample, and what does this not cause an error, and why doesn't it do anything?
How could I solve this to make my set-23 function work?
I also had the idea to use a thunk for this to defer execution of aref, just like:
(defun set-23 (fn)
(setf (funcall fn) 23))
But this already runs into an error when I try to define this function, as Lisp now tells me:
(SETF FUNCALL) is only defined for functions of the form #'symbol.
Again, I wonder why this is. Why does using setf in combination with funcall apparently work for named functions, but not for lambdas, e.g.?
PS: In "Land of Lisp" (which I'm currently reading to learn about Lisp) it says:
In fact, the first argument in setf is a special sublanguage of Common Lisp, called a generalized reference. Not every Lisp command is allowed in a generalized reference, but you can still put in some pretty complicated stuff: […]
Well, I guess that this is the reason (or at least one of the reasons) here, why all this does not work as I'd expect it, but nevertheless I'm curious to learn more :-)
A place is nothing physical, it's just a concept for anything where we can get/set a value. So a place in general can't be returned or passed. Lisp developers wanted a way to easily guess a setter from just knowing what the getter is. So we write the getter, with a surrounding setf form and Lisp figures out how to set something:
(slot-value vehicle 'speed) ; gets the speed
(setf (slot-value vehicle 'speed) 100) ; sets the speed
Without SETF we would need a setter function with its name:
(set-slot-value vehicle 'speed 100) ; sets the speed
For setting an array we would need another function name:
(set-aref 3d-board 100 100 100 'foo) ; sets the board at 100/100/100
Note that the above setter functions might exist internally. But you don't need to know them with setf.
Result: we end up with a multitude of different setter function names.
The SETF mechanism replaces ALL of them with one common syntax. You know the getter call? Then you know the setter, too. It's just setf around the getter call plus the new value.
Another example
world-time ; may return the world time
(setf world-time (get-current-time)) ; sets the world time
And so on...
Note also that only macros deal with setting places: setf, push, pushnew, remf, ... Only with those you can set a place.
(defun set-23 (place)
(setf place 23))
Above can be written, but place is just a variable name. You can't pass a place. Let's rename it, which does not change a thing, but reduces confusion:
(defun set-23 (foo)
(setf foo 23))
Here foo is a local variable. A local variable is a place. Something we can set. So we can use setf to set the local value of the variable. We don't set something that gets passed in, we set the variable itself.
(defmethod set-24 ((vehicle audi-vehicle))
(setf (vehicle-speed vehicle) 100))
In above method, vehicle is a variable and it is bound to an object of class audi-vehicle. To set the speed of it, we use setf to call the writer method.
Where does Lisp know the writer from? For example a class declaration generates one:
(defclass audi-vehicle ()
((speed :accessor vehicle-speed)))
The :accessor vehicle-speed declaration causes both reading and setting functions to be generated.
The setf macro looks at macro expansion time for the registered setter. That's all. All setf operations look similar, but Lisp underneath knows how to set things.
Here are some examples for SETF uses, expanded:
Setting an array item at an index:
CL-USER 86 > (pprint (macroexpand-1 '(setf (aref a1 10) 'foo)))
(LET* ((#:G10336875 A1) (#:G10336876 10) (#:|Store-Var-10336874| 'FOO))
(SETF::\"COMMON-LISP\"\ \"AREF\" #:|Store-Var-10336874|
#:G10336875
#:G10336876))
Setting a variable:
CL-USER 87 > (pprint (macroexpand-1 '(setf a 'foo)))
(LET* ((#:|Store-Var-10336877| 'FOO))
(SETQ A #:|Store-Var-10336877|))
Setting a CLOS slot:
CL-USER 88 > (pprint (macroexpand-1 '(setf (slot-value o1 'bar) 'foo)))
(CLOS::SET-SLOT-VALUE O1 'BAR 'FOO)
Setting the first element of a list:
CL-USER 89 > (pprint (macroexpand-1 '(setf (car some-list) 'foo)))
(SYSTEM::%RPLACA SOME-LIST 'FOO)
As you can see it uses a lot of internal code in the expansion. The user just writes a SETF form and Lisp figures out what code would actually do the thing.
Since you can write your own setter, only your imagination limits the things you might want to put under this common syntax:
setting a value on another machine via some network protocol
setting some value in a custom data structure you've just invented
setting a value in a database
In your example:
(defun set-23 (place)
(setf place 23))
you can't do it just like that, because you have to use setf in context.
This will work:
(defmacro set-23 (place)
`(setf ,place 23))
CL-USER> (set-23 (aref *foo* 0))
23
CL-USER> *foo*
#(23 NIL NIL NIL NIL)
The trick is, setf 'knows' how to look at real place its arguments come from, only for limited number of functions. These functions are called setfable.
setf is a macro, and to use it the way you wanted to, you also have to use macros.
The reason why you have not been getting errors, is that you actually successfully modified lexical variable place which was bound to copy of selected array element.
Related
I want to make a macro for binding variables to values given a var-list and a val-list.
This is my code for it -
(defmacro let-bind (vars vals &body body)
`(let ,(loop for x in vars
for y in vals
collect `(,x ,y))
,#body))
While it works correct if called like (let-bind (a b) (1 2) ...), it doesn't seem to work when called like
(defvar vars '(a b))
(defvar vals '(1 2))
(let-bind vars vals ..)
Then I saw some effects for other of my macros too. I am a learner and cannot find what is wrong.
Basic problem: a macro sees code, not values. A function sees values, not code.
CL-USER 2 > (defvar *vars* '(a b))
*VARS*
CL-USER 3 > (defvar *vals* '(1 2))
*VALS*
CL-USER 4 > (defmacro let-bind (vars vals &body body)
(format t "~%the value of vars is: ~a~%" vars)
`(let ,(loop for x in vars
for y in vals
collect `(,x ,y))
,#body))
LET-BIND
CL-USER 5 > (let-bind *vars* *vals* t)
the value of vars is: *VARS*
Error: *VARS* (of type SYMBOL) is not of type LIST.
1 (abort) Return to top loop level 0.
You can see that the value of vars is *vars*. This is a symbol. Because the macro variables are bound to code fragments - not their values.
Thus in your macro you try to iterate over the symbol *vars*. But *vars* is a symbol and not a list.
You can now try to evaluate the symbol *vars* at macro expansion time. But that won't work also in general, since at macro expansion time *vars* may not have a value.
Your macro expands into a let form, but let expects at compile time real variables. You can't compute the variables for let at a later point in time. This would work only in some interpreted code where macros would be expanded at runtime - over and over.
If you’ve read the other answers then you know that you can’t read a runtime value from a compiletime macro (or rather, you can’t know the value it will have at runtime at compiletime as you can’t see the future). So let’s ask a different question: how can you bind the variables in your list known at runtime.
In the case where your list isn’t really variable and you just want to give it a single name you could use macroexpand:
(defun symbol-list-of (x env)
(etypecase x
(list x)
(symbol (macroexpand x env))))
(defmacro let-bind (vars vals &body body &environment env)
(let* ((vars (symbol-list-of vars env))
(syms (loop for () in vars collect gensym)))
`(destructuring-bind ,syms ,vals
(let ,(loop for sym in syms for bar in vars collect (list var sym)) ,#body))))
This would somewhat do what you want. It will symbol-macroexpand the first argument and evaluate the second.
What if you want to evaluate the first argument? Well we could try generating something that uses eval. As eval will evaluate in the null lexical environment (ie can’t refer to any external local variables), we would need to have eval generate a function to bind variables and then call another function. That is a function like (lambda (f) (let (...) (funcall f)). You would evaluate the expression to get that function and then call it with a function which does he body (but was not made by eval and so captures the enclosing scope). Note that this would mean that you could only bind dynamic variables.
What if you want to bind lexical variables? Well there is no way to go from symbol to the memory location of a variable at runtime in Common Lisp. A debugger might know how to do this. There is no way to get a list of variables in scope in a macro, although the compiler knows this. So you can’t generate a function to set a lexically bound symbol. And it would be even harder to do if you wanted to shadow the binding although you could maybe do it with some symbol-macrolet trickery if you knew every variable in scope.
But maybe there is a better way to do this for special variables and it turns out there is. It’s an obscure special form called progv. It has the same signature that you want let-bind to have except it works. link.
Why is my variable nodes undefined in the vector-push-extend line?
(defun make_graph (strings)
(defparameter nodes (make-array 0))
(loop for x in strings do
(vector-push-extend (make-instance 'node :data x) nodes))
n)
The short answer is that you should use let instead of defparameter to introduce your variable. For instance:
(defun make_graph (strings)
(let ((nodes (make-array 0)))
(loop for x in strings do
(vector-push-extend (make-instance 'node :data x) nodes))
;; your code says N here, but I assume that's a typo...
nodes))
The defparameter form is useful for creating "special" variables, which are somewhat similar to global variables in other programming languages. (There are some differences, e.g., the special variables introduced by defparameter aren't exactly global---instead, they are dynamically scoped, and can be let bound, etc...)
At any rate, the let form will instead create a local variable.
DEFPARAMETER is used at toplevel to define global special variables.
Toplevel:
(defparameter *foo* 42)
Still at toplevel, because forms inside PROGN are still at toplevel (by definition):
(progn
(defparameter *foo* 42)
(defparameter *bar* 32))
Not at toplevel:
(defun baz ()
(defparameter *foo* 42))
Above last form is not recognized by the compiler as a variable declaration. But when one calls (baz) and the function is running, the variable is defined and initialized.
A non-toplevel use of DEFPARAMETER will not be recognized by the compiler, but at runtime it will create a special global variable.
(defun make_graph (strings)
(defparameter nodes (make-array 0))
(loop for x in strings do
(vector-push-extend (make-instance 'node :data x) nodes))
n)
The compiler warns:
;;;*** Warning in MAKE_GRAPH: NODES assumed special
;;;*** Warning in MAKE_GRAPH: N assumed special
Thus in above code, the compiler does not recognize nodes as a defined variable, if it wasn't defined somewhere else already. The use of nodes in the function creates a warning.
Still the code might work, since at runtime the variable is created and initialized - but for every function invocation. Over and over. This compiler also assumes that nodes is just this: some kind of special variable. Still I would not count on it for all compilers.
n is also not defined anywhere.
Notes:
the correct way to introduce local lexical variables is to use LET and LET* (and other binding forms)
use DEFPARAMETER as a toplevel form. It is unusual when it's not a toplevel form. Typically the author makes a mistake then.
I'm currently reading the book Land of LISP, and I'm just working through the first chapter. In there, there is a little program written where the computer guesses numbers between 1 and 100. Its code is as follows:
(defparameter *small* 1)
(defparameter *big* 100)
(defun guess-my-number ()
(ash (+ *small* *big*) -1))
(defun smaller ()
(setf *big* (1- (guess-my-number)))
(guess-my-number))
(defun bigger ()
(setf *small* (1+ (guess-my-number)))
(guess-my-number))
(defun start-over ()
(defparameter *small* 1)
(defparameter *big* 100)
(guess-my-number))
So far, I understand what happens, and Using 'ash' in LISP to perform a binary search? helped me a lot in this. Nevertheless there's one thing left that puzzles me: As far as I have learned, you use setf to assign values to variables, and defparameter to initially define variables. I also have understood the difference between defparameter and defvar(at least I believe I do ;-)).
So now my question is: If I should use setf to assign a value to a variable once it had been initialized, why does the start-over function use defparameter and not setf? Is there a special reason for this, or is this just sloppiness?
The function is just:
(defun start-over ()
(setf *small* 1)
(setf *big* 100)
(guess-my-number))
It is already declared to be a special global variable. No need to do it inside the function again and again.
You CAN use DEFPARAMETER inside a function, but it is bad style.
DEFPARAMETER is for declaring global special variables and optional documentation for them. Once. If you need to do it several times, it's mostly done when a whole file or system gets reloaded. The file compiler also recognizes it in top-level position as a special declaration for a dynamically bound variable.
Example:
File 1:
(defparameter *foo* 10)
(defun foo ()
(let ((*foo* ...))
...))
File 2:
(defun foo-start ()
(defparameter *foo* 10))
(defun foo ()
(let ((*foo* ...))
...))
If Lisp compiles File 1 fresh with compile-file, the compiler recognizes the defparameter and in the following let we have a dynamic binding.
If Lisp compiles File 2 fresh with compile-file, the compiler doesn't recognize the defparameter and in the following let we have a lexical binding. If we compile it again, from this state, we have a dynamic binding.
So here version 1 is better, because it is easier to control and understand.
In your example DEFPARAMETER appears multiple times, which is not useful. I might ask, where is the variable defined and the answer would point to multiple source locations...
So: make sure that your program elements get mostly defined ONCE - unless you have a good reason not to do so.
So you have global variables. Those can be defined by defconstant (for really non-chainging stuff), defparameter (a constant that you can change) and defvar (a variable that does not overwrite if you load.
You use setf to alter the state of lexical as well and global variables. start-over could have used setf since the code doesn't really define it but change it. If you would have replaced defparameter with defvar start-over would stop working.
(defparameter *par* 5)
(setf *par* 6)
(defparameter *par* 5)
*par* ; ==> 5
(defvar *var* 5)
(setf *var* 6)
(defvar *var* 5)
*var* ; ==> 6
defconstant is like defparameter except once defined the CL implementation is free to inline it in code. Thus if you redefine a constant you need to redefine all functions that uses it or else it might use the old value.
(defconstant +c+ 5)
(defun test (x)
(+ x +c+))
(test 1) ; ==> 6
(defconstant +c+ 6)
(test 1) ; ==> 6 or 7
(defun test (x)
(+ x +c+))
(test 1) ; ==> 7
Normally, one would use defvar to initialy define global variables. The difference between defvar and defparameter is subtle, cf. the section in the CLHS and this plays a role here: defparameter (in contrast to defvar) assigns the value anew, whereas defvar would leave the old binding in place.
To address what to use: In general, defvar and friends are used as top-level forms, not inside some function (closures being the most notable exception in the context of defun). I would use setf, not defparameter.
For a beginner symbols, variables, etc. can be a bit surprising. Symbols are surprisingly featureful. Just to mention a few things you can ask a symbol for it's symbol-value, symbol-package, symbol-name, symbol-function etc. In addition symbols can have a varying amount of information declared about them, for example a type, that provides advice the compile might leverage to create better code. This is true of all symbols, for example *, the symbol you use for multiplication has a symbol-function that does that multiplication. It also has a symbol-value, i.e. the last value returned in the current REPL.
One critical bit of declarative information about symbols is if they are "special." (Yeah, it's a dumb name.) For good reasons it's good practice to declare all global symbols to be special. defvar, defparameter, and defconstant do that for you. We have a convention that all special variables are spelled with * on the front and back, *standard-output* for example. This convention is so common that some compilers will warn you if you neglect to follow it.
One benefit of declaring a symbol as special is that it will suppress the warning you get when you misspell a variable in your functions. For example (defun faster (how-much) (incf *speed* hw-much)) will generate a warning about hw-much.
The coolest feature of a symbol that is special is that it is managed with what we call dynamic scoping, in contrast to lexical scope. Recall how * has the value of the last result in the REPL. Well in some implementations you can have multiple REPLs (each running in it's own thread) each will want to have it's own *, it's own *standard-output*, etc. etc. This is easy in Common Lisp; the threads establish a "dynamic extent" and bind the specials that should be local to that REPL.
So yes, you could just setf *small* in you example; but if you never declare it to be special then you are going to get warnings about how the compiler thinks you misspelled it.
Can someone help me understand how push can be implemented as a macro? The naive version below evaluates the place form twice, and does so before evaluating the element form:
(defmacro my-push (element place)
`(setf ,place (cons ,element ,place)))
But if I try to fix this as below then I'm setf-ing the wrong place:
(defmacro my-push (element place)
(let ((el-sym (gensym))
(place-sym (gensym)))
`(let ((,el-sym ,element)
(,place-sym ,place))
(setf ,place-sym (cons ,el-sym ,place-sym)))))
CL-USER> (defparameter *list* '(0 1 2 3))
*LIST*
CL-USER> (my-push 'hi *list*)
(HI 0 1 2 3)
CL-USER> *list*
(0 1 2 3)
How can I setf the correct place without evaluating twice?
Doing this right seems to be a little more complicated. For instance, the code for push in SBCL 1.0.58 is:
(defmacro-mundanely push (obj place &environment env)
#!+sb-doc
"Takes an object and a location holding a list. Conses the object onto
the list, returning the modified list. OBJ is evaluated before PLACE."
(multiple-value-bind (dummies vals newval setter getter)
(sb!xc:get-setf-expansion place env)
(let ((g (gensym)))
`(let* ((,g ,obj)
,#(mapcar #'list dummies vals)
(,(car newval) (cons ,g ,getter))
,#(cdr newval))
,setter))))
So reading the documentation on get-setf-expansion seems to be useful.
For the record, the generated code looks quite nice:
Pushing into a symbol:
(push 1 symbol)
expands into
(LET* ((#:G906 1) (#:NEW905 (CONS #:G906 SYMBOL)))
(SETQ SYMBOL #:NEW905))
Pushing into a SETF-able function (assuming symbol points to a list of lists):
(push 1 (first symbol))
expands into
(LET* ((#:G909 1)
(#:SYMBOL908 SYMBOL)
(#:NEW907 (CONS #:G909 (FIRST #:SYMBOL908))))
(SB-KERNEL:%RPLACA #:SYMBOL908 #:NEW907))
So unless you take some time to study setf, setf expansions and company, this looks rather arcane (it may still look so even after studying them). The 'Generalized Variables' chapter in OnLisp may be useful too.
Hint: if you compile your own SBCL (not that hard), pass the --fancy argument to make.sh. This way you'll be able to quickly see the definitions of functions/macros inside SBCL (for instance, with M-. inside Emacs+SLIME). Obviously, don't delete those sources (you can run clean.sh after install.sh, to save 90% of the space).
Taking a look at how the existing one (in SBCL, at least) does things, I see:
* (macroexpand-1 '(push 1 *foo*))
(LET* ((#:G823 1) (#:NEW822 (CONS #:G823 *FOO*)))
(SETQ *FOO* #:NEW822))
T
So, I imagine, mixing in a combination of your version and what this generates, one might do:
(defmacro my-push (element place)
(let ((el-sym (gensym))
(new-sym (gensym "NEW")))
`(let* ((,el-sym ,element)
(,new-sym (cons ,el-sym ,place)))
(setq ,place ,new-sym)))))
A few observations:
This seems to work with either setq or setf. Depending on what problem you're actually trying to solve (I presume re-writing push isn't the actual end goal), you may favor one or the other.
Note that place does still get evaluated twice... though it does at least do so only after evaluating element. Is the double evaluation something you actually need to avoid? (Given that the built-in push doesn't, I'm left wondering if/how you'd be able to... though I'm writing this up before spending terribly much time thinking about it.) Given that it's something that needs to evaluate as a "place", perhaps this is normal?
Using let* instead of let allows us to use ,el-sym in the setting of ,new-sym. This moves where the cons happens, such that it's evaluated in the first evaluation of ,place, and after the evaluation of ,element. Perhaps this gets you what you need, with respect to evaluation ordering?
I think the biggest problem with your second version is that your setf really does need to operate on the symbol passed in, not on a gensym symbol.
Hopefully this helps... (I'm still somewhat new to all this myself, so I'm making some guesses here.)
Consider this piece of code:
(defvar lst '(1 1))
(defmacro get-x (x lst)
`(nth ,x ,lst))
(defun get-y (y lst)
(nth y lst))
Now let us assume that I want to change the value of the elements of the list called lst, the car with get-x and the cdr with get-y.
As I try to change the value with get-x (with setf) everything goes fine but if I try it with get-y it signals an error (shortened):
; caught STYLE-WARNING:
; undefined function: (SETF GET-STUFF)
Why does this happen?
I myself suspect that this happens because the macro simply expands and the function nth simply returns a reference to the value of an element in the list and the function on the other hand evaluates the function-call to nth and returns the value of the referenced value (sounds confusing).
Am I correct in my suspicions?
If I am correct then how will one know what is simply a reference to a value and an actual value?
The error does not happen with the macro version, because, as you assumed, the expression (setf (get-x some-x some-list) some-value) will be expanded (at compile-time) into something like (setf (nth some-x some-list) some-value) (not really, but the details of setf-expansion are complex), and the compiler knows, how to deal with that (i.e., there is a suitable setf expander defined for function nth).
However, in the case of get-y, the compiler has no setf expander, unless you provide one. The easiest way to do so would be
(defun (setf get-y) (new-value x ls) ; Note the function's name: setf get-y
(setf (nth x ls) new-value))
Note, that there are a few conventions regarding setf-expanders:
The new value is always provided as the first argument to the setf function
All setf functions are supposed to return the new value as their result (as this is, what the entire setf form is supposed to return)
There is, BTW, no such concept as a "reference" in Common Lisp (at least not in the C++ sense), though there once were Lisp dialects which had locatives. Generalized place forms (ie., setf and its machinery) work very differently from plain C++ style references. See the CLHS, if you are curious about the details.
SETF is a macro.
The idea is that to set and read elements from data structures are two operations, but usually require two different names (or maybe even something more complex). SETF now enables you to use just one name for both:
(get-something x)
Above reads a datastructure. The inverse then simply is:
(setf (get-something x) :foobar)
Above sets the datastructure at X with :FOOBAR.
SETF does not treat (get-something x) as a reference or something like that. It just has a database of inverse operations for each operation. If you use GET-SOMETHING, it knows what the inverse operation is.
How does SETF know it? Simple: you have to tell it.
For The NTH operation, SETF knows how to set the nth element. That's builtin into Common Lisp.
For your own GET-Y operation SETF does not have that information. You have to tell it. See the Common Lisp HyperSpec for examples. One example is to use DEFUN and (SETF GET-Y) as a function name.
Also note following style problems with your example:
lst is not a good name for a DEFVAR variable. Use *list* as a name to make clear that it is a special variable declared by DEFVAR (or similar).
'(1 2) is a literal constant. If you write a Common Lisp program, the effects of changing it are undefined. If you want to change a list later, you should cons it with LIST or something like COPY-LIST.