What's happening when the special variable is declared during compile time - lisp

I just met an unusual situation in my common lisp code when I wanna test locally and declare:
(defvar test-out 2) ;; make a dynamic variable
;; function below just simply re-write from locally doc
(defun test (out)
(declare (special out))
(let ((out 1))
(print out) ;; => 1
(print (locally (declare (special out)) out)))) ;; => 2
;; when argument has same name as outside dynamic variable
(defun test1 (test-out)
(declare (special test-out))
(let ((test-out 1))
(print test-out) ;; => 1
(print (locally (declare (special test-out)) test-out)))) ;; => also 1
I know the right name of dynamic variable should be *test-out*, but I thought it just for programmer to tell dynamic variable conveniently.
I am a bit confuse about test1 function, it looks like locally declare do not point test-out to dynamic variable outside.
Can anyone explain to me test1 function's behavior? Thanks
Update:
I give a new dynamic variable (defvar test-out-1 3), and call it like (test1 test-out-1), still get print out result 1 and 1.
I change test1's argument name from test-out to test-out1, recompile test1, and problem disappears, print out results are 1 and 2 when I call (test1 test-out).
I change (defvar test-out 2) to (defvar test-out-1 2) (change dynamic variable name). Then recompile whole file (no dynamic variable called test-out this time, and test1 argument's name is test-out), problem disappears.
After 3, I call (defvar test-out 2), and (test1 test-out). This time, it prints out right answers: 1 and 2.
After 4, I recompile test1 again, then run (test1 test-out), it prints out 1 and 1 again, problem appears again.
If I guess right, when test1 compiling, for some reason, its arguments' name connect to dynamic variable test-out. That's why I receiving wrong result when I even call with different value, however, problem is solved by itself when I recompile test1 with different argument name or clean dynamic variable test-out before recompile test.
If so, I still do not understand why compile function will effect by dynamic variable in the environment.

DEFVAR declares a variable to be special - that means they will use dynamic bindings when bound and access to such a variable will look for dynamic bindings. Globally and on all binding levels. For now and in the future.
From then on, ALL uses and bindings of that variable in new code will be declared special automatically. Even local LET bindings. On all levels. There is no way to declare it unspecial. Thus a local special declaration in your test1 function now is not needed, it already is declared special. Every use or binding of it, even without explicit declaration is now using dynamic binding.
That's also the reason why any DEFVAR or DEFPARAMETER variable should be written as *variablename*, to prevent accidentally declaring all variables with the same name to be special.
Avoid:
(defvar x 10) ; here X is explicitly declared special
(defun foo (x) ; here X is implicitly declared special
(let ((x ...)) ; here X is implicitly declared special
...))
Do:
(defvar *x* 10) ; here *X* is declared special
(defun foo (x) ; here X is lexical
(let ((x ...)) ; here X is lexical
...))

Related

Using special variables as macro input?

I want to make a macro for binding variables to values given a var-list and a val-list.
This is my code for it -
(defmacro let-bind (vars vals &body body)
`(let ,(loop for x in vars
for y in vals
collect `(,x ,y))
,#body))
While it works correct if called like (let-bind (a b) (1 2) ...), it doesn't seem to work when called like
(defvar vars '(a b))
(defvar vals '(1 2))
(let-bind vars vals ..)
Then I saw some effects for other of my macros too. I am a learner and cannot find what is wrong.
Basic problem: a macro sees code, not values. A function sees values, not code.
CL-USER 2 > (defvar *vars* '(a b))
*VARS*
CL-USER 3 > (defvar *vals* '(1 2))
*VALS*
CL-USER 4 > (defmacro let-bind (vars vals &body body)
(format t "~%the value of vars is: ~a~%" vars)
`(let ,(loop for x in vars
for y in vals
collect `(,x ,y))
,#body))
LET-BIND
CL-USER 5 > (let-bind *vars* *vals* t)
the value of vars is: *VARS*
Error: *VARS* (of type SYMBOL) is not of type LIST.
1 (abort) Return to top loop level 0.
You can see that the value of vars is *vars*. This is a symbol. Because the macro variables are bound to code fragments - not their values.
Thus in your macro you try to iterate over the symbol *vars*. But *vars* is a symbol and not a list.
You can now try to evaluate the symbol *vars* at macro expansion time. But that won't work also in general, since at macro expansion time *vars* may not have a value.
Your macro expands into a let form, but let expects at compile time real variables. You can't compute the variables for let at a later point in time. This would work only in some interpreted code where macros would be expanded at runtime - over and over.
If you’ve read the other answers then you know that you can’t read a runtime value from a compiletime macro (or rather, you can’t know the value it will have at runtime at compiletime as you can’t see the future). So let’s ask a different question: how can you bind the variables in your list known at runtime.
In the case where your list isn’t really variable and you just want to give it a single name you could use macroexpand:
(defun symbol-list-of (x env)
(etypecase x
(list x)
(symbol (macroexpand x env))))
(defmacro let-bind (vars vals &body body &environment env)
(let* ((vars (symbol-list-of vars env))
(syms (loop for () in vars collect gensym)))
`(destructuring-bind ,syms ,vals
(let ,(loop for sym in syms for bar in vars collect (list var sym)) ,#body))))
This would somewhat do what you want. It will symbol-macroexpand the first argument and evaluate the second.
What if you want to evaluate the first argument? Well we could try generating something that uses eval. As eval will evaluate in the null lexical environment (ie can’t refer to any external local variables), we would need to have eval generate a function to bind variables and then call another function. That is a function like (lambda (f) (let (...) (funcall f)). You would evaluate the expression to get that function and then call it with a function which does he body (but was not made by eval and so captures the enclosing scope). Note that this would mean that you could only bind dynamic variables.
What if you want to bind lexical variables? Well there is no way to go from symbol to the memory location of a variable at runtime in Common Lisp. A debugger might know how to do this. There is no way to get a list of variables in scope in a macro, although the compiler knows this. So you can’t generate a function to set a lexically bound symbol. And it would be even harder to do if you wanted to shadow the binding although you could maybe do it with some symbol-macrolet trickery if you knew every variable in scope.
But maybe there is a better way to do this for special variables and it turns out there is. It’s an obscure special form called progv. It has the same signature that you want let-bind to have except it works. link.

Why is my variable "undefined"?

Why is my variable nodes undefined in the vector-push-extend line?
(defun make_graph (strings)
(defparameter nodes (make-array 0))
(loop for x in strings do
(vector-push-extend (make-instance 'node :data x) nodes))
n)
The short answer is that you should use let instead of defparameter to introduce your variable. For instance:
(defun make_graph (strings)
(let ((nodes (make-array 0)))
(loop for x in strings do
(vector-push-extend (make-instance 'node :data x) nodes))
;; your code says N here, but I assume that's a typo...
nodes))
The defparameter form is useful for creating "special" variables, which are somewhat similar to global variables in other programming languages. (There are some differences, e.g., the special variables introduced by defparameter aren't exactly global---instead, they are dynamically scoped, and can be let bound, etc...)
At any rate, the let form will instead create a local variable.
DEFPARAMETER is used at toplevel to define global special variables.
Toplevel:
(defparameter *foo* 42)
Still at toplevel, because forms inside PROGN are still at toplevel (by definition):
(progn
(defparameter *foo* 42)
(defparameter *bar* 32))
Not at toplevel:
(defun baz ()
(defparameter *foo* 42))
Above last form is not recognized by the compiler as a variable declaration. But when one calls (baz) and the function is running, the variable is defined and initialized.
A non-toplevel use of DEFPARAMETER will not be recognized by the compiler, but at runtime it will create a special global variable.
(defun make_graph (strings)
(defparameter nodes (make-array 0))
(loop for x in strings do
(vector-push-extend (make-instance 'node :data x) nodes))
n)
The compiler warns:
;;;*** Warning in MAKE_GRAPH: NODES assumed special
;;;*** Warning in MAKE_GRAPH: N assumed special
Thus in above code, the compiler does not recognize nodes as a defined variable, if it wasn't defined somewhere else already. The use of nodes in the function creates a warning.
Still the code might work, since at runtime the variable is created and initialized - but for every function invocation. Over and over. This compiler also assumes that nodes is just this: some kind of special variable. Still I would not count on it for all compilers.
n is also not defined anywhere.
Notes:
the correct way to introduce local lexical variables is to use LET and LET* (and other binding forms)
use DEFPARAMETER as a toplevel form. It is unusual when it's not a toplevel form. Typically the author makes a mistake then.

When to use defparameter instead of setf?

I'm currently reading the book Land of LISP, and I'm just working through the first chapter. In there, there is a little program written where the computer guesses numbers between 1 and 100. Its code is as follows:
(defparameter *small* 1)
(defparameter *big* 100)
(defun guess-my-number ()
(ash (+ *small* *big*) -1))
(defun smaller ()
(setf *big* (1- (guess-my-number)))
(guess-my-number))
(defun bigger ()
(setf *small* (1+ (guess-my-number)))
(guess-my-number))
(defun start-over ()
(defparameter *small* 1)
(defparameter *big* 100)
(guess-my-number))
So far, I understand what happens, and Using 'ash' in LISP to perform a binary search? helped me a lot in this. Nevertheless there's one thing left that puzzles me: As far as I have learned, you use setf to assign values to variables, and defparameter to initially define variables. I also have understood the difference between defparameter and defvar(at least I believe I do ;-)).
So now my question is: If I should use setf to assign a value to a variable once it had been initialized, why does the start-over function use defparameter and not setf? Is there a special reason for this, or is this just sloppiness?
The function is just:
(defun start-over ()
(setf *small* 1)
(setf *big* 100)
(guess-my-number))
It is already declared to be a special global variable. No need to do it inside the function again and again.
You CAN use DEFPARAMETER inside a function, but it is bad style.
DEFPARAMETER is for declaring global special variables and optional documentation for them. Once. If you need to do it several times, it's mostly done when a whole file or system gets reloaded. The file compiler also recognizes it in top-level position as a special declaration for a dynamically bound variable.
Example:
File 1:
(defparameter *foo* 10)
(defun foo ()
(let ((*foo* ...))
...))
File 2:
(defun foo-start ()
(defparameter *foo* 10))
(defun foo ()
(let ((*foo* ...))
...))
If Lisp compiles File 1 fresh with compile-file, the compiler recognizes the defparameter and in the following let we have a dynamic binding.
If Lisp compiles File 2 fresh with compile-file, the compiler doesn't recognize the defparameter and in the following let we have a lexical binding. If we compile it again, from this state, we have a dynamic binding.
So here version 1 is better, because it is easier to control and understand.
In your example DEFPARAMETER appears multiple times, which is not useful. I might ask, where is the variable defined and the answer would point to multiple source locations...
So: make sure that your program elements get mostly defined ONCE - unless you have a good reason not to do so.
So you have global variables. Those can be defined by defconstant (for really non-chainging stuff), defparameter (a constant that you can change) and defvar (a variable that does not overwrite if you load.
You use setf to alter the state of lexical as well and global variables. start-over could have used setf since the code doesn't really define it but change it. If you would have replaced defparameter with defvar start-over would stop working.
(defparameter *par* 5)
(setf *par* 6)
(defparameter *par* 5)
*par* ; ==> 5
(defvar *var* 5)
(setf *var* 6)
(defvar *var* 5)
*var* ; ==> 6
defconstant is like defparameter except once defined the CL implementation is free to inline it in code. Thus if you redefine a constant you need to redefine all functions that uses it or else it might use the old value.
(defconstant +c+ 5)
(defun test (x)
(+ x +c+))
(test 1) ; ==> 6
(defconstant +c+ 6)
(test 1) ; ==> 6 or 7
(defun test (x)
(+ x +c+))
(test 1) ; ==> 7
Normally, one would use defvar to initialy define global variables. The difference between defvar and defparameter is subtle, cf. the section in the CLHS and this plays a role here: defparameter (in contrast to defvar) assigns the value anew, whereas defvar would leave the old binding in place.
To address what to use: In general, defvar and friends are used as top-level forms, not inside some function (closures being the most notable exception in the context of defun). I would use setf, not defparameter.
For a beginner symbols, variables, etc. can be a bit surprising. Symbols are surprisingly featureful. Just to mention a few things you can ask a symbol for it's symbol-value, symbol-package, symbol-name, symbol-function etc. In addition symbols can have a varying amount of information declared about them, for example a type, that provides advice the compile might leverage to create better code. This is true of all symbols, for example *, the symbol you use for multiplication has a symbol-function that does that multiplication. It also has a symbol-value, i.e. the last value returned in the current REPL.
One critical bit of declarative information about symbols is if they are "special." (Yeah, it's a dumb name.) For good reasons it's good practice to declare all global symbols to be special. defvar, defparameter, and defconstant do that for you. We have a convention that all special variables are spelled with * on the front and back, *standard-output* for example. This convention is so common that some compilers will warn you if you neglect to follow it.
One benefit of declaring a symbol as special is that it will suppress the warning you get when you misspell a variable in your functions. For example (defun faster (how-much) (incf *speed* hw-much)) will generate a warning about hw-much.
The coolest feature of a symbol that is special is that it is managed with what we call dynamic scoping, in contrast to lexical scope. Recall how * has the value of the last result in the REPL. Well in some implementations you can have multiple REPLs (each running in it's own thread) each will want to have it's own *, it's own *standard-output*, etc. etc. This is easy in Common Lisp; the threads establish a "dynamic extent" and bind the specials that should be local to that REPL.
So yes, you could just setf *small* in you example; but if you never declare it to be special then you are going to get warnings about how the compiler thinks you misspelled it.

define a symbol local inside the function?

Just as an example, put inside the function create x property and its value globally:
(defun foo ()
(put 'spam 'x 1))
(foo)
(get 'spam 'x) ; -> 1
Is it there way to set the symbol property locally?
No, because 'spam is always the same symbol a property can't be set on it locally.
I don't know if this would be appropriate for your situation, but you could create a fresh symbol and put the property on that. Because the symbol wouldn't be available outside the function neither would the property.
(defun foo ()
(let ((private (make-symbol "private")))
(put private 'x 1)
(get private 'x)))
(foo) ;=> 1
(get 'private 'x) ;=> nil
make-symbol returns a "newly allocated [and] uninterned symbol", which means the symbol returned by (make-symbol "private") is a different symbol from the global 'private and all others. See here for the Emacs manual's section on creating and interning symbols for more information.
Emacs also supports buffer-local variables, though that's not quite the same thing (the symbol's value is local to a particular buffer, but the symbol itself and its properties are still global).
If you just need to bind a value to a name locally, you could also use either Emacs 24's support for lexical binding or, if you're on an older version, lexical-let from the cl package (which is included with Emacs).
You can do it "locally" in the sense of dynamic-scoping:
(require 'cl-lib)
(defun foo ()
(cl-letf (((get 'spam 'x) 1))
(get 'spam 'x)))
(foo) ; -> 1
(get 'spam 'x) ; -> nil
Although I don't quite understand what is that you want to do, seems to me that you are looking for a closure, that is, a function with an environment. To do so you have to enable lexical binding, which is supported starting from emacs 24.3 IIRC. To enable it set the buffer local variable lexical-binding to t. The popular example of a closure would be an adder factory, that is a function that returns a function that adds by a constant.
(defun make-adder (constant)
(lambda (y) (+ y constant)))
(make-adder 3)
;; As you can see a closure is a function with an environment associated
=> (closure ((constant . 3) t) (y) (+ y constant))
(funcall (make-adder 3) 2)
=> 5
(funcall (make-adder 5) 2)
=> 8
So yes, using closures you can have private variables for a function.

Difference between LET and SETQ?

I'm programming on Ubuntu using GCL. From the documentation on Common Lisp from various sources, I understand that let creates local variables, and setq sets the values of existing variables. In cases below, I need to create two variables and sum their values.
Using setq
(defun add_using_setq ()
(setq a 3) ; a never existed before , but still I'm able to assign value, what is its scope?
(setq b 4) ; b never existed before, but still I'm able to assign value, what is its scope?
(+ a b))
Using let
(defun add_using_let ( )
(let ((x 3) (y 4)) ; creating variables x and y
(+ x y)))
In both the cases I seem to achieve the same result; what is the difference between using setq and let here? Why can't I use setq (since it is syntactically easy) in all the places where I need to use let?
setq assigns a value to a variable, whereas let introduces new variables/bindings. E.g., look what happens in
(let ((x 3))
(print x) ; a
(let ((x 89))
(print x) ; b
(setq x 73)
(print x)) ; c
(print x)) ; d
3 ; a
89 ; b
73 ; c
3 ; d
The outer let creates a local variable x, and the inner let creates another local variable shadowing the inner one. Notice that using let to shadow the variable doesn't affect the shadowed variable's value; the x in line d is the x introduced by the outer let, and its value hasn't changed. setq only affects the variable that it is called with. This example shows setq used with local variables, but it can also be with special variables (meaning, dynamically scoped, and usually defined with defparameter or defvar:
CL-USER> (defparameter *foo* 34)
*FOO*
CL-USER> (setq *foo* 93)
93
CL-USER> *foo*
93
Note that setq doesn't (portably) create variables, whereas let, defvar, defparameter, &c. do. The behavior of setq when called with an argument that isn't a variable (yet) isn't defined, and it's up to an implementation to decide what to do. For instance, SBCL complains loudly:
CL-USER> (setq new-x 89)
; in: SETQ NEW-X
; (SETQ NEW-X 89)
;
; caught WARNING:
; undefined variable: NEW-X
;
; compilation unit finished
; Undefined variable:
; NEW-X
; caught 1 WARNING condition
89
Of course, the best ways to get a better understanding of these concepts are to read and write more Lisp code (which comes with time) and to read the entries in the HyperSpec and follow the cross references, especially the glossary entries. E.g., the short descriptions from the HyperSpec for setq and let include:
SETQ
Assigns values to variables.
LET
let and let* create new variable bindings and execute a series
of forms that use these bindings.
You may want to read more about variables and bindings. let and let* also have some special behavior with dynamic variables and special declarations (but you probably won't need to know about that for a while), and in certain cases (that you probably won't need to know about for a while) when a variable isn't actually a variable, setq is actually equivalent to setf. The HyperSpec has more details.
There are some not-quite duplicate questions on Stack Overflow that may, nonetheless, help in understanding the use of the various variable definition and assignment operators available in Common Lisp:
setq and defvar in lisp
What's difference between defvar, defparameter, setf and setq
Assigning variables with setf, defvar, let and scope
In Lisp, how do I fix "Warning: Assumed Special?" (re: using setq on undefined variables)
Difference between let* and set? in Common Lisp
Let should almost always be the way you bind variables inside of a function definition -- except in the rare case where you want the value to be available to other functions in the same scope.
I like the description in the emacs lisp manual:
let is used to attach or bind a symbol to a value in such a way that the Lisp interpreter will not confuse the variable with a variable of the same name that is not part of the function.
To understand why the let special form is necessary, consider the situation in which you own a home that you generally refer to as ‘the house,’ as in the sentence, “The house needs painting.” If you are visiting a friend and your host refers to ‘the house,’ he is likely to be referring to his house, not yours, that is, to a different house.
If your friend is referring to his house and you think he is referring to your house, you may be in for some confusion. The same thing could happen in Lisp if a variable that is used inside of one function has the same name as a variable that is used inside of another function, and the two are not intended to refer to the same value. The let special form prevents this kind of confusion.
-- http://www.gnu.org/software/emacs/manual/html_node/eintr/let.html
(setq x y) assigns a new value y to the variable designated by the symbol x, optionally defining a new package-level variable 1. This means that after you called add_using_setq you will have two new package-level variables in the current package.
(add_using_setq)
(format t "~&~s, ~s" a b)
will print 3 4 - unlikely the desired outcome.
To contrast that, when you use let, you only assign new values to variables designated by symbols for the duration of the function, so this code will result in an error:
(add_using_let)
(format t "~&~s, ~s" a b)
Think about let as being equivalent to the following code:
(defun add-using-lambda ()
(funcall (lambda (a b) (+ a b)) 3 4))
As an aside, you really want to look into code written by other programmers to get an idea of how to name or format things. Beside being traditional it also has some typographic properties you don't really want to loose.
1 This behaviour is non-standard, but this is what happens in many popular implementations. Regardless of it being fairly predictable, it is considered a bad practice for other reasons, mostly all the same concerns that would discourage you from using global variables.
SETQ
you can get the value of the symbol out of the scope, as long as Lisp is still running. (it assign the value to the symbol)
LET
you can't get the value of the symbol defined with LET after Lisp has finished evaluating the form. (it bind the value to the symbol and create new binding to symbol)
consider example below:
;; with setq
CL-USER> (setq a 10)
CL-USER> a
10
;; with let
CL-USER> (let ((b 20))
(print b))
CL-USER> 20
CL-USER> b ; it will fail
; Evaluation aborted on #<UNBOUND-VARIABLE B {1003AC1563}>.
CL-USER>