This question already has answers here:
setq and defvar in Lisp
(4 answers)
Closed 6 years ago.
I was following a tutorial on lisp and they did the following code
(set 'x 11)
(incf x 10)
and interpreter gave the following error:
; in: INCF X
; (SETQ X #:NEW671)
;
; caught WARNING:
; undefined variable: X
;
; compilation unit finished
; Undefined variable:
; X
; caught 1 WARNING condition
21
what is the proper way to increment x ?
This is indeed how you are meant to increment x, or at least one way of doing so. However it is not how you are meant to bind x. In CL you need to establish a binding for a name before you use it, and you don't do that by just assigning to it. So, for instance, this code (in a fresh CL image) is not legal CL:
(defun bad ()
(setf y 2))
Typically this will cause a compile-time warning and a run-time error, although it may do something else: its behaviour is not defined.
What you have done, in particular, is actually worse than this: you have rammed a value into the symbol-value of x (with set, which does this), and then assumed that something like (incf x) will work, which it is extremely unlikely to do. For instance consider something like this:
(defun worse ()
(let ((x 2))
(set 'x 4)
(incf x)
(values x (symbol-value 'x))))
This is (unlike bad) legal code, but it probably does not do what you want it to do.
Many CL implementations do allow assignment to previously unbound variables at the top-level, because in a conversational environment it is convenient. But the exact meaning of such assignments is outwith the language standard.
CMUCL and its derivatives, including SBCL, have historically been rather more serious about this than other implementations were at the time. I think the reason for this is that the interpreter was a bunch more serious than most others and/or they secretly compiled everything anyway and the compiler picked things up.
A further problem is that CL has slightly awkward semantics for top-level variables: if you go to the effort to establish a toplevel binding in the normal way, with defvar & friends, then you also cause the variable to be special -- dynamically scoped -- and this is a pervasive effect: it makes all bindings of that name special. That is often a quite undesirable consequence. CL, as a language, has no notion of a top-level lexical variable.
What many implementations did, therefore, was to have some kind of informal notion of a top-level binding of something which did not imply a special declaration: if you just said (setf x 3) at the toplevel then this would not contage the entire environment. But then there were all sorts of awkward questions: after doing that, what is the result of (symbol-value 'x) for instance?
Fortunately CL is a powerful language, and it is quite possible to define top-level lexical variables within the language. Here is a very hacky implementation called deflexical. Note that there are better implementations out there (including at least one by me, which I can't find right now): this is not meant to be a bullet-proof solution.
(defmacro deflexical (var &optional value)
;; Define a cheap-and-nasty global lexical variable. In this
;; implementation, global lexicals are not boundp and the global
;; lexical value is not stored in the symbol-value of the symbol.
;;
;; This implementation is *not* properly thought-through and is
;; without question problematic
`(progn
(define-symbol-macro ,var (get ',var 'lexical-value))
(let ((flag (cons nil nil)))
;; assign a value only if there is not one already, like DEFVAR
(when (eq (get ',var 'lexical-value flag) flag)
(setf (get ',var 'lexical-value) ,value))
;; Return the symbol
',var)))
Related
I'm a total Lisp n00b, so please be gentle.
I'm having trouble wrapping my head around CL's idea of an [un-]declared free variable. I would think that:
(defun test ()
(setq foo 17)
)
would define a function that declares a variable foo and set it to 17. However, instead I get
;Compiler warnings :
; In TEST: Undeclared free variable FOO
My actual example case is a bit bigger; my code (snippet) looks like this:
(defun p8 ()
;;; [some other stuff, snip]
(loop for x from 0 to (- (length str) str-len) do
(setq last (+ x str-len)) ; get the last char of substring
(setq subs (subseq str x last)) ; get the substring
(setq prod (prod-string subs)) ; get the product of that substring
(if (> prod max) ; if it's bigger than current max, save it
(setq max prod)
(setq max-str subs)
)
)
;;; [More stuff, snip]
)
and that gives me:
;Compiler warnings for "/path/to/Lisp/projectEuler/p6-10.lisp":
; In P8: Undeclared free variable LAST (2 references)
;Compiler warnings for "/Volumes/TwoBig/AllYourBits-Olie/WasOnDownBelowTheOcean/zIncoming/Lisp/projectEuler/p6-10.lisp" :
; In P8: Undeclared free variable PROD (3 references)
;Compiler warnings for "/Volumes/TwoBig/AllYourBits-Olie/WasOnDownBelowTheOcean/zIncoming/Lisp/projectEuler/p6-10.lisp" :
; In P8: Undeclared free variable SUBS (3 references)
;Compiler warnings for "/Volumes/TwoBig/AllYourBits-Olie/WasOnDownBelowTheOcean/zIncoming/Lisp/projectEuler/p6-10.lisp" :
; In P8: Undeclared free variable =
Yes, yes, I realize that I'm using far too many intermediate variables, but I'm trying to understand what's going on before I get too fancy with compressing everything down to the minimal typed characters, as seems popular in the CL world.
So, anyway... can someone explain the following:
Under what conditions does Lisp "declare" a variable?
Is the scope of said variable other than the enclosing (...) around the setq statement?! (That is, I would expect the var to be valid & scoped for everything from (... (setq ...) ...) the parens 1 level outside the setq, no?
Am I mis-interpreting the Undeclared free variable message?
Any other hints you care to give that would help me better grok what's going on, here.
NOTE: I'm fairly expert with C, Java, Javascript, Obj-C, and related procedural languages. I get that functional programming is different. For now, I'm just wrestling with the syntax.
Thanks!
P.S. If it matters, defun p8 is in a text file (TextMate) and I'm running it on Clozure CL. Hopefully, none of that matters, though!
In lisp variables may be declared usingdefparameterordefvar.
(defparameter var1 5)
(defvar var2 42)
This results in global (dynamic) variables.
The difference between defvarand defparameteris that defvardoes not reinitialize an already existing variable.
Local (lexical) variables are introduced e.g using let or let* (which initializes the variables sequentially).
Undeclared free variable means that you have used (here setq-ed) a variable that it is not bound in the context it is used. It may then be declared for you, but then probably as a global (dynamic) variable. The consequence of this is that if you use undeclared variables with the same name in several functions, you will be referencing the same variable in all of the functions.
Your code can be written like this:
(loop for x from 0 to (- (length str) str-len) do
(let* ((last (+ x str-len)) ; get the last char of substring
(subs (subseq str x last)) ; get the substring
(prod (prod-string subs))) ; get the product of that substring
(if (> prod max) ; if it's bigger than current max, save it
(setq max prod)
(setq max-str subs))))
Using the variable-binding properties of loop, it may also be written as
(loop for x from 0 to (- (length str) str-len)
for last = (+ x str-len)
for subs = (subseq str x last)
for prod = (prod-string subs)
when (> prod max) do
(setq max prod)
(setq max-str subs))
In Lisp variable declaration may be performed in many ways. The most notable are:
declaring global (they are properly called special) variables with defparameter and defvar
declaring local variables with let, let*, multiple-value-bind, destructuring-bind, and other binding forms
as function arguments
You can read about their scope in many places as well, for instance in CLtL2.
setq/setf are not variable declaration operators, but variable modification operators, as implied by their names.
PS. In interactive mode some implementation would use the DWIM approach and declare the variable as special behind the scenes, if you try to set an undeclared variable, but this is purely for convenience.
The Common Lisp HyperSpec (basically its the Common Lisp standard in HTML form) says:
http://www.lispworks.com/documentation/HyperSpec/Body/s_setq.htm
Assigns values to variables.
So SETQ only assigns values to variables. It does not declare them.
Variable definitions are done globally with DEFVAR, DEFPARAMETER, ...
(defparameter *this-is-a-global-dynamic-variable* 'yep)
Variable definitions are done locally with DEFUN, LET, LET*, LOOP, and many others.
(defun foo (v1 v2)
...)
(let ((v1 10)
(v2 20))
...)
(loop for v1 in '(10 30 10 20)
do ...)
This is basic Lisp and it would be useful to read an introduction. I would recommend:
http://www.cs.cmu.edu/~dst/LispBook/
Above book is free for download.
Additionally the above mentioned Common Lisp Hyperspec provides you with the definitions for Common Lisp and describes the various facilities (DEFUN, LOOP, DEFPARAMETER, ...) in detail.
I'm currently reading the book Land of LISP, and I'm just working through the first chapter. In there, there is a little program written where the computer guesses numbers between 1 and 100. Its code is as follows:
(defparameter *small* 1)
(defparameter *big* 100)
(defun guess-my-number ()
(ash (+ *small* *big*) -1))
(defun smaller ()
(setf *big* (1- (guess-my-number)))
(guess-my-number))
(defun bigger ()
(setf *small* (1+ (guess-my-number)))
(guess-my-number))
(defun start-over ()
(defparameter *small* 1)
(defparameter *big* 100)
(guess-my-number))
So far, I understand what happens, and Using 'ash' in LISP to perform a binary search? helped me a lot in this. Nevertheless there's one thing left that puzzles me: As far as I have learned, you use setf to assign values to variables, and defparameter to initially define variables. I also have understood the difference between defparameter and defvar(at least I believe I do ;-)).
So now my question is: If I should use setf to assign a value to a variable once it had been initialized, why does the start-over function use defparameter and not setf? Is there a special reason for this, or is this just sloppiness?
The function is just:
(defun start-over ()
(setf *small* 1)
(setf *big* 100)
(guess-my-number))
It is already declared to be a special global variable. No need to do it inside the function again and again.
You CAN use DEFPARAMETER inside a function, but it is bad style.
DEFPARAMETER is for declaring global special variables and optional documentation for them. Once. If you need to do it several times, it's mostly done when a whole file or system gets reloaded. The file compiler also recognizes it in top-level position as a special declaration for a dynamically bound variable.
Example:
File 1:
(defparameter *foo* 10)
(defun foo ()
(let ((*foo* ...))
...))
File 2:
(defun foo-start ()
(defparameter *foo* 10))
(defun foo ()
(let ((*foo* ...))
...))
If Lisp compiles File 1 fresh with compile-file, the compiler recognizes the defparameter and in the following let we have a dynamic binding.
If Lisp compiles File 2 fresh with compile-file, the compiler doesn't recognize the defparameter and in the following let we have a lexical binding. If we compile it again, from this state, we have a dynamic binding.
So here version 1 is better, because it is easier to control and understand.
In your example DEFPARAMETER appears multiple times, which is not useful. I might ask, where is the variable defined and the answer would point to multiple source locations...
So: make sure that your program elements get mostly defined ONCE - unless you have a good reason not to do so.
So you have global variables. Those can be defined by defconstant (for really non-chainging stuff), defparameter (a constant that you can change) and defvar (a variable that does not overwrite if you load.
You use setf to alter the state of lexical as well and global variables. start-over could have used setf since the code doesn't really define it but change it. If you would have replaced defparameter with defvar start-over would stop working.
(defparameter *par* 5)
(setf *par* 6)
(defparameter *par* 5)
*par* ; ==> 5
(defvar *var* 5)
(setf *var* 6)
(defvar *var* 5)
*var* ; ==> 6
defconstant is like defparameter except once defined the CL implementation is free to inline it in code. Thus if you redefine a constant you need to redefine all functions that uses it or else it might use the old value.
(defconstant +c+ 5)
(defun test (x)
(+ x +c+))
(test 1) ; ==> 6
(defconstant +c+ 6)
(test 1) ; ==> 6 or 7
(defun test (x)
(+ x +c+))
(test 1) ; ==> 7
Normally, one would use defvar to initialy define global variables. The difference between defvar and defparameter is subtle, cf. the section in the CLHS and this plays a role here: defparameter (in contrast to defvar) assigns the value anew, whereas defvar would leave the old binding in place.
To address what to use: In general, defvar and friends are used as top-level forms, not inside some function (closures being the most notable exception in the context of defun). I would use setf, not defparameter.
For a beginner symbols, variables, etc. can be a bit surprising. Symbols are surprisingly featureful. Just to mention a few things you can ask a symbol for it's symbol-value, symbol-package, symbol-name, symbol-function etc. In addition symbols can have a varying amount of information declared about them, for example a type, that provides advice the compile might leverage to create better code. This is true of all symbols, for example *, the symbol you use for multiplication has a symbol-function that does that multiplication. It also has a symbol-value, i.e. the last value returned in the current REPL.
One critical bit of declarative information about symbols is if they are "special." (Yeah, it's a dumb name.) For good reasons it's good practice to declare all global symbols to be special. defvar, defparameter, and defconstant do that for you. We have a convention that all special variables are spelled with * on the front and back, *standard-output* for example. This convention is so common that some compilers will warn you if you neglect to follow it.
One benefit of declaring a symbol as special is that it will suppress the warning you get when you misspell a variable in your functions. For example (defun faster (how-much) (incf *speed* hw-much)) will generate a warning about hw-much.
The coolest feature of a symbol that is special is that it is managed with what we call dynamic scoping, in contrast to lexical scope. Recall how * has the value of the last result in the REPL. Well in some implementations you can have multiple REPLs (each running in it's own thread) each will want to have it's own *, it's own *standard-output*, etc. etc. This is easy in Common Lisp; the threads establish a "dynamic extent" and bind the specials that should be local to that REPL.
So yes, you could just setf *small* in you example; but if you never declare it to be special then you are going to get warnings about how the compiler thinks you misspelled it.
I've written an ad hoc parser generator that creates code to convert an old and little known 7-bit character set into unicode. The call to the parser generator expands into a bunch of defuns enclosed in a progn, which then get compiled. I only want to expose one of the generated defuns--the top-level one--to the rest of the system; all the others are internal to the parser and only get called from within the dynamic scope of the top-level one. Therefore, the other defuns generated have uninterned names (created with gensym). This strategy works fine with SBCL, but I recently tested it for the first time with CLISP, and I get errors like:
*** - FUNCALL: undefined function #:G16985
It seems that CLISP can't handle functions with uninterned names. (Interestingly enough, the system compiled without a problem.) EDIT: It seems that it can handle functions with uninterned names in most cases. See the answer by Rörd below.
My questions is: Is this a problem with CLISP, or is it a limitation of Common Lisp that certain implementations (e.g. SBCL) happen to overcome?
EDIT:
For example, the macro expansion of the top-level generated function (called parse) has an expression like this:
(PRINC (#:G75735 #:G75731 #:G75733 #:G75734) #:G75732)
Evaluating this expression (by calling parse) causes an error like the one above, even though the function is definitely defined within the very same macro expansion:
(DEFUN #:G75735 (#:G75742 #:G75743 #:G75744) (DECLARE (OPTIMIZE (DEBUG 2)))
(DECLARE (LEXER #:G75742) (CONS #:G75743 #:G75744))
(MULTIPLE-VALUE-BIND (#:G75745 #:G75746) (POP-TOKEN #:G75742)
...
The two instances of #:G75735 are definitely the same symbol--not two different symbols with the same name. As I said, this works with SBCL, but not with CLISP.
EDIT:
SO user Joshua Taylor has pointed out that this is due to a long standing CLISP bug.
You don't show one of the lines that give you the error, so I can only guess, but the only thing that could cause this problem as far as I can see is that you are referring to the name of the symbol instead of the symbol itself when trying to call it.
If you were referring to the symbol itself, all your lisp implementation would have to do is lookup that symbol's symbol-function. Whether it's interned or not couldn't possibly matter.
May I ask why you haven't considered another way to hide the functions, i.e. a labels statement or defining the functions within a new package that exports only the one external function?
EDIT: The following example is copied literally from an interaction with the CLISP prompt.
As you can see, calling the function named by a gensym is working as expected.
[1]> (defmacro test ()
(let ((name (gensym)))
`(progn
(defun ,name () (format t "Hello!"))
(,name))))
TEST
[2]> (test)
Hello!
NIL
Maybe your code that's trying to call the function gets evaluated before the defun? If there's any code in the macro expansion besides the various defuns, it may be implementation-dependent what gets evaluated first, and so the behaviour of SBCL and CLISP may differ without any of them violating the standard.
EDIT 2: Some further investigation shows that CLISP's behaviour varies depending upon whether the code is interpreted directly or whether it's first compiled and then interpreted. You can see the difference by either directly loading a Lisp file in CLISP or by first calling compile-file on it and then loading the FASL.
You can see what's going on by looking at the first restart that CLISP offers. It says something like "Input a value to be used instead of (FDEFINITION '#:G3219)." So for compiled code, CLISP quotes the symbol and refers to it by name.
It seems though that this behaviour is standard-conforming. The following definition can be found in the HyperSpec:
function designator n. a designator for a function; that is, an object that denotes a function and that is one of: a symbol (denoting the function named by that symbol in the global environment), or a function (denoting itself). The consequences are undefined if a symbol is used as a function designator but it does not have a global definition as a function, or it has a global definition as a macro or a special form. See also extended function designator.
I think an uninterned symbol matches the "a symbol is used as a function designator but it does not have a global definition as a function" case for unspecified consequences.
EDIT 3: (I can agree that I'm not sure whether CLISP's behaviour is a bug or not. Someone more experienced with details of the standard's terminology should judge this. It comes down to whether the function cell of an uninterned symbol - i.e. a symbol that cannot be referred to by name, only by having a direct hold on the symbol object - would be considered a "global definition" or not)
Anyway, here's an example solution that solves the problem in CLISP by interning the symbols in a throwaway package, avoiding the matter of uninterned symbols:
(defmacro test ()
(let* ((pkg (make-package (gensym)))
(name (intern (symbol-name (gensym)) pkg)))
`(progn
(defun ,name () (format t "Hello!"))
(,name))))
(test)
EDIT 4: As Joshua Taylor notes in a comment to the question, this seems to be a case of the (10 year old) CLISP bug #180.
I've tested both workarounds suggested in that bug report and found that replacing the progn with locally actually doesn't help, but replacing it with let () does.
You can most certainly define functions whose names are uninterned symbols. For instance:
CL-USER> (defun #:foo (x)
(list x))
#:FOO
CL-USER> (defparameter *name-of-function* *)
*NAME-OF-FUNCTION*
CL-USER> *name-of-function*
#:FOO
CL-USER> (funcall *name-of-function* 3)
(3)
However, the sharpsign colon syntax introduces a new symbol each time such a form is read read:
#: introduces an uninterned symbol whose name is symbol-name. Every time this syntax is encountered, a distinct uninterned symbol is created. The symbol-name must have the syntax of a symbol with no package prefix.
This means that even though something like
CL-USER> (list '#:foo '#:foo)
;=> (#:FOO #:FOO)
shows the same printed representation, you actually have two different symbols, as the following demonstrates:
CL-USER> (eq '#:foo '#:foo)
NIL
This means that if you try to call such a function by typing #: and then the name of the symbol naming the function, you're going to have trouble:
CL-USER> (#:foo 3)
; undefined function #:foo error
So, while you can call the function using something like the first example I gave, you can't do this last one. This can be kind of confusing, because the printed representation makes it look like this is what's happening. For instance, you could write such a factorial function like this:
(defun #1=#:fact (n &optional (acc 1))
(if (zerop n) acc
(#1# (1- n) (* acc n))))
using the special reader notation #1=#:fact and #1# to later refer to the same symbol. However, look what happens when you print that same form:
CL-USER> (pprint '(defun #1=#:fact (n &optional (acc 1))
(if (zerop n) acc
(#1# (1- n) (* acc n)))))
(DEFUN #:FACT (N &OPTIONAL (ACC 1))
(IF (ZEROP N)
ACC
(#:FACT (1- N) (* ACC N))))
If you take that printed output, and try to copy and paste it as a definition, the reader creates two symbols named "FACT" when it comes to the two occurrences of #:FACT, and the function won't work (and you might even get undefined function warnings):
CL-USER> (DEFUN #:FACT (N &OPTIONAL (ACC 1))
(IF (ZEROP N)
ACC
(#:FACT (1- N) (* ACC N))))
; in: DEFUN #:FACT
; (#:FACT (1- N) (* ACC N))
;
; caught STYLE-WARNING:
; undefined function: #:FACT
;
; compilation unit finished
; Undefined function:
; #:FACT
; caught 1 STYLE-WARNING condition
I hope I get the issue right. For me it works in CLISP.
I tried it like this: using a macro for creating a function with a GENSYM-ed name.
(defmacro test ()
(let ((name (gensym)))
`(progn
(defun ,name (x) (* x x))
',name)))
Now I can get the name (setf x (test)) and call it (funcall x 2).
Yes, it is perfectly fine defining functions that have names that are unintenred symbols. The problem is that you cannot then call them "by name", since you can't fetch the uninterned symbol by name (that is what "uninterned" means, essentially).
You would need to store the uninterned symbol in some sort of data structure, to then be able to fetch the symbol. Alternatively, store the defined function in some sort of data structure.
Surprisingly, CLISP bug 180 isn't actually an ANSI CL conformance bug. Not only that, but evidently, ANSI Common Lisp is itself so broken in this regard that even the progn based workaround is a courtesy of the implementation.
Common Lisp is a language intended for compilation, and compilation produces issues regarding the identity of objects which are placed into compiled files and later loaded ("externalized" objects). ANSI Common Lisp requires that literal objects reproduced from compiled files are only similar to the original objects. (CLHS 3.2.4 Literal Objects in Compiled Files).
Firstly, according to the definition similarity (3.2.4.2.2 Definition of Similarity), the rules for uninterned symbols is that similarity is name based. If we compile code with a literal that contains an uninterned symbol, then when we load the compiled file, we get a symbol which is similar and not (necessarily) the same object: a symbol which has the same name.
What if the same uninterned symbol is inserted into two different top-level forms which are then compiled as a file? When the file is loaded, are those two similar to each other at least? No, there is no such requirement.
But it gets worse: there is also no requirement that two occurrences of the same uninterned symbol in the same form will be externalized in such a way that their relative identity is preserved: that the re-loaded version of that object will have the same symbol object in all the places where the original was. In fact, the definition of similarity contains no provision for preserving the circular structure and substructure sharing. If we have a literal like '#1=(a b . #1#), as a literal in a compiled file, there appears to be no requirement that this be reproduced as a circular object with the same graph structure as the original (a graph isomorphism). The similarity rule for conses is given as naive recursion: two conses are similar if their respective cars and cdrs are similar. (The rule can't even be evaluated for circular objects; it doesn't terminate).
That the above works is because of implementations going beyond what is required in the spec; they are providing an extension consistent with (3.2.4.3 Extensions to Similarity Rules).
Thus, purely according to ANSI CL, we cannot expect to use macros with gensyms in compiled files, at least in some ways. The expectation expressed in code like the following runs afoul of the spec:
(defmacro foo (arg)
(let ((g (gensym))
(literal '(blah ,g ,g ,arg)))
...))
(defun bar ()
(foo 42))
The bar function contains a literal with two insertions of a gensym, which according to the similarity rules for conses and symbols need not reproduce as a list containing two occurrences of the same object in the second and third positions.
If the above works as expected, it's due to "extensions to the similarity rules".
So the answer to the "Why can't CLISP ..." question is that although CLISP does provide an extension for similarity which preserves the graph structure of literal forms, it doesn't do it across the entire compiled file, only within individual top level items within that file. (It uses *print-circle* to emit the individual items.) The bug is that CLISP doesn't conform to the best possible behavior users can imagine, or at least to a better behavior exhibited by other implementations.
I'm programming on Ubuntu using GCL. From the documentation on Common Lisp from various sources, I understand that let creates local variables, and setq sets the values of existing variables. In cases below, I need to create two variables and sum their values.
Using setq
(defun add_using_setq ()
(setq a 3) ; a never existed before , but still I'm able to assign value, what is its scope?
(setq b 4) ; b never existed before, but still I'm able to assign value, what is its scope?
(+ a b))
Using let
(defun add_using_let ( )
(let ((x 3) (y 4)) ; creating variables x and y
(+ x y)))
In both the cases I seem to achieve the same result; what is the difference between using setq and let here? Why can't I use setq (since it is syntactically easy) in all the places where I need to use let?
setq assigns a value to a variable, whereas let introduces new variables/bindings. E.g., look what happens in
(let ((x 3))
(print x) ; a
(let ((x 89))
(print x) ; b
(setq x 73)
(print x)) ; c
(print x)) ; d
3 ; a
89 ; b
73 ; c
3 ; d
The outer let creates a local variable x, and the inner let creates another local variable shadowing the inner one. Notice that using let to shadow the variable doesn't affect the shadowed variable's value; the x in line d is the x introduced by the outer let, and its value hasn't changed. setq only affects the variable that it is called with. This example shows setq used with local variables, but it can also be with special variables (meaning, dynamically scoped, and usually defined with defparameter or defvar:
CL-USER> (defparameter *foo* 34)
*FOO*
CL-USER> (setq *foo* 93)
93
CL-USER> *foo*
93
Note that setq doesn't (portably) create variables, whereas let, defvar, defparameter, &c. do. The behavior of setq when called with an argument that isn't a variable (yet) isn't defined, and it's up to an implementation to decide what to do. For instance, SBCL complains loudly:
CL-USER> (setq new-x 89)
; in: SETQ NEW-X
; (SETQ NEW-X 89)
;
; caught WARNING:
; undefined variable: NEW-X
;
; compilation unit finished
; Undefined variable:
; NEW-X
; caught 1 WARNING condition
89
Of course, the best ways to get a better understanding of these concepts are to read and write more Lisp code (which comes with time) and to read the entries in the HyperSpec and follow the cross references, especially the glossary entries. E.g., the short descriptions from the HyperSpec for setq and let include:
SETQ
Assigns values to variables.
LET
let and let* create new variable bindings and execute a series
of forms that use these bindings.
You may want to read more about variables and bindings. let and let* also have some special behavior with dynamic variables and special declarations (but you probably won't need to know about that for a while), and in certain cases (that you probably won't need to know about for a while) when a variable isn't actually a variable, setq is actually equivalent to setf. The HyperSpec has more details.
There are some not-quite duplicate questions on Stack Overflow that may, nonetheless, help in understanding the use of the various variable definition and assignment operators available in Common Lisp:
setq and defvar in lisp
What's difference between defvar, defparameter, setf and setq
Assigning variables with setf, defvar, let and scope
In Lisp, how do I fix "Warning: Assumed Special?" (re: using setq on undefined variables)
Difference between let* and set? in Common Lisp
Let should almost always be the way you bind variables inside of a function definition -- except in the rare case where you want the value to be available to other functions in the same scope.
I like the description in the emacs lisp manual:
let is used to attach or bind a symbol to a value in such a way that the Lisp interpreter will not confuse the variable with a variable of the same name that is not part of the function.
To understand why the let special form is necessary, consider the situation in which you own a home that you generally refer to as ‘the house,’ as in the sentence, “The house needs painting.” If you are visiting a friend and your host refers to ‘the house,’ he is likely to be referring to his house, not yours, that is, to a different house.
If your friend is referring to his house and you think he is referring to your house, you may be in for some confusion. The same thing could happen in Lisp if a variable that is used inside of one function has the same name as a variable that is used inside of another function, and the two are not intended to refer to the same value. The let special form prevents this kind of confusion.
-- http://www.gnu.org/software/emacs/manual/html_node/eintr/let.html
(setq x y) assigns a new value y to the variable designated by the symbol x, optionally defining a new package-level variable 1. This means that after you called add_using_setq you will have two new package-level variables in the current package.
(add_using_setq)
(format t "~&~s, ~s" a b)
will print 3 4 - unlikely the desired outcome.
To contrast that, when you use let, you only assign new values to variables designated by symbols for the duration of the function, so this code will result in an error:
(add_using_let)
(format t "~&~s, ~s" a b)
Think about let as being equivalent to the following code:
(defun add-using-lambda ()
(funcall (lambda (a b) (+ a b)) 3 4))
As an aside, you really want to look into code written by other programmers to get an idea of how to name or format things. Beside being traditional it also has some typographic properties you don't really want to loose.
1 This behaviour is non-standard, but this is what happens in many popular implementations. Regardless of it being fairly predictable, it is considered a bad practice for other reasons, mostly all the same concerns that would discourage you from using global variables.
SETQ
you can get the value of the symbol out of the scope, as long as Lisp is still running. (it assign the value to the symbol)
LET
you can't get the value of the symbol defined with LET after Lisp has finished evaluating the form. (it bind the value to the symbol and create new binding to symbol)
consider example below:
;; with setq
CL-USER> (setq a 10)
CL-USER> a
10
;; with let
CL-USER> (let ((b 20))
(print b))
CL-USER> 20
CL-USER> b ; it will fail
; Evaluation aborted on #<UNBOUND-VARIABLE B {1003AC1563}>.
CL-USER>
The reason I don't understand why Emacs 24's new lexical scoping features are that great is that I can't think of any new functionality that couldn't have been implemented without them. For example, the following closure:
(setq lexical-binding t)
(defun f1 (num1)
(lambda (num2)
(setq num1 (* num1 num2))))
(fset 'f2 (f1 5))
==> (closure ((num1 . 5) t) (num2) (setq num1 (* num1 num2)))
(f2 5)
==> 25
(f2 2)
==> 50
Can be implemented with regular dynamic scoping like so:
(defun f1 (num)
(let ((tmpvar (make-symbol "num")))
(set tmpvar num)
`(lambda (num2)
(set ',tmpvar (* (eval ,tmpvar) num2)))))
(fset 'f2 (f1 5))
==> (lambda (num2) (set (quote num) (+ (eval num) num2)))
(f2 5)
==> 25
(f2 2)
==> 50
(fset 'f3 (f1 9))
==> (lambda (num2) (set (quote num) (+ (eval num) num2)))
(f3 3)
==> 27
(f3 2)
==> 54
(f2 10)
==> 500
Okay, so not all languages have something analogous to elisp's uninterned symbols, so I understand why lexical scoping is so great in their case. But what about elisp? Can you think of anything that I can do now (as of Emacs 24) that I couldn't do before, thanks to lexical scoping?
You don't need uninterned symbols, use cons instead of make-symbol, car instead of eval, and setcar instead of set and your example will work just as well (and be more efficient).
Note also that the progression from machine-language to ever higher-level languages has been mostly based on making more and more things impossible (or at least much harder). Of course those facilities taken away from the programmers were rarely used and/or considered too dangerous. Think of using uninitialized variables (possible in C, but impossible in Java and many other languages), or jumping into the middle of an instruction.
As for some downside of your example code: not only it's less readable, but the compiler basically won't be able to know that you're constructing code, so it won't be allowed to look inside that "`(lambda ...)" to compile it, expand its macro calls, give you warnings about suspicious elements, ...
There have always been workarounds to simulate lexical binding in Emacs, so it's not so much about being able to do new things.
The manual says:
Lexical binding opens up a lot more opportunities for optimization, so
Emacs Lisp code that makes use of lexical binding is likely to run
faster in future Emacs versions. Such code is also much more friendly
to concurrency, which we want to add to Emacs in the near future.
I think that is the primary benefit.
The flip side to this is that dynamic binding was purposely chosen when Emacs was written for good reasons which still hold true today, and so lexical binding should certainly not be considered the New Way of doing things.
Global variables are generally considered a bad idea in programming, but Emacs is an unusual case where this doesn't really apply, because much of its immense flexibility -- one of the key things that makes Emacs great -- derives directly from dynamic binding. Without dynamic binding, it would not be possible to bend the application to the requirements of the individual user to anything like the extent that Emacs allows.
In my opinion, lexical binding should be used cautiously and only for variables for which another user could not conceivably find a reason to override. By default variables should be defvar'd so that the ability to customize the behaviour (even in ways that the author did not anticipate) is preserved.
I think implementations of OO programming tend to fit well with lexical scope. An object with state maps rather directly to a lexical closure.
I'm sure that the CLOS implementation in common lisp leverages lexical scope heavily. It's hard for me to imagine how that spec could be implemented with dynamic scope only, but I'm sure it's possible.