In Lisp, why do we need to use a list function to return a list? - lisp

I would have thought this would work:
(defun list-of-things (thing1 thing2 thing3)
"returns a ???"
'(thing1 thing2 thing3))
But this is actually what is needed to return a list:
(defun list-of-things (thing1 thing2 thing3)
"returns a list"
(list thing1 thing2 thing3))

Those are two different things, and a central concept in Lisp languages.
The syntax '(a b c) is a shorthand for (quote (a b c)) and quote is a special form that returns its argument unevaluated. In that case, it would be list, containing three symbols.
On the other hand, list is a normal function, it evaluates its arguments and return the list of their values.
To go a bit more in-depth: before evaluating any expression, the code has to be read. The part that is very specific to Lisp (and, in fact, even more so with Common Lisp) is that the "parser" is actually just a Lisp function, returning Lisp objects. Those objects are then given to the "evaluator", which evaluates them:
When reading the characters '(thing1 thing2 thing3), the reader knows that ' is a reader-macro, and so it reads (quote (thing1 thing2 thing3)). This is a list of two elements, a symbol and another list of three symbols. This list is then given to the evaluator: it knows that quote is a special form that returns its argument unevaluated, and so it simply returns the list (thing1 thing2 thing3), given to it by the reader.
On the other hand, when reading (list thing1 thing2 thing3), the reader also reads this as a list (this time, containing 4 symbols) that it then gives to the evaluator. Now, the evaluator sees that the first symbol is list, a function, and so it evaluates the arguments (i.e. it determines what the symbols thing1 ... are bound to), passes them to the list functions, etc.
All this is possible because Lisp code is defined in terms of Lisp objects (such as lists, etc). After the "parsing" (actually called reading) phase, the evaluator is really given an actual Lisp object to evaluate. Things are of course slightly more complicated when talking about compilation, etc, but the general idea is the same. The quote operator, abbreviated ', is a way to "directly" access the thing created by the reader, and to "bypass" evaluation.

Lisp evaluation rules
You need to understand the evaluation rules of Common Lisp first:
the symbol a is a variable and evaluates to its value
the list (operator ... ) is a form. There are four types of forms: function call, macro form, special form, lambda form
everything else evaluates to itself: numbers, strings, arrays, hash tables, characters, CLOS objects, structures, ...
There is a fixed number of special operators. One is QUOTE. (QUOTE ...) means that the object inside does not get evaluated. '(...) is another notation for (QUOTE ...) - it's easier to write.
So symbols and lists have a role: they are variables and forms.
What if you want to have literal symbols and literal lists? Answer: you have to quote them.
Remember: If you quote a list, then the whole list including its elements is literal data and not evaluated.
Examples:
A global variable defined:
CL-USER 18 > (defvar *a* 42)
*A*
The symbol *a* is quoted and thus the form (QUOTE *A*) evaluates to the symbol:
CL-USER 19 > '*a*
*A*
Inside quoted lists everything is literal data and not evaluated.
CL-USER 20 > '(*a*) ; same as (QUOTE (*A*))
(*A*)
The unquoted symbol is evaluated as a variable:
CL-USER 21 > *a*
42
A function call has all its arguments evaluated:
CL-USER 22 > (list *a*)
(42)
A backquote expression allows certain elements in a list to be evaluated using the comma prefix:
CL-USER 23 > `(41 ,*a* (+ 1 *a*) (+ 1 ,*a*) ,(+ 2 *a*))
(41 42 (+ 1 *A*) (+ 1 42) 44)
Summary
When you want to return a list from a function, you can:
return a fixed literal list
or compute a new list based on some objects by constructing the list with the usual operators: CONS, LIST, APPEND, ...
or use a backquote expression which gets expanded by the Lisp reader into an implementation specific version of 2.

The list function is used for creating lists. They called LISP because it’s for LISt Processing. Also, because Lisp functions are written as lists, they can be processed exactly like list.

Related

LISP: when modifying a list (with `nth`), all elements change [duplicate]

After making it through the major parts of an introductory Lisp book, I still couldn't understand what the special operator (quote) (or equivalent ') function does, yet this has been all over Lisp code that I've seen.
What does it do?
Short answer
Bypass the default evaluation rules and do not evaluate the expression (symbol or s-exp), passing it along to the function exactly as typed.
Long Answer: The Default Evaluation Rule
When a regular (I'll come to that later) function is invoked, all arguments passed to it are evaluated. This means you can write this:
(* (+ a 2)
3)
Which in turn evaluates (+ a 2), by evaluating a and 2. The value of the symbol a is looked up in the current variable binding set, and then replaced. Say a is currently bound to the value 3:
(let ((a 3))
(* (+ a 2)
3))
We'd get (+ 3 2), + is then invoked on 3 and 2 yielding 5. Our original form is now (* 5 3) yielding 15.
Explain quote Already!
Alright. As seen above, all arguments to a function are evaluated, so if you would like to pass the symbol a and not its value, you don't want to evaluate it. Lisp symbols can double both as their values, and markers where you in other languages would have used strings, such as keys to hash tables.
This is where quote comes in. Say you want to plot resource allocations from a Python application, but rather do the plotting in Lisp. Have your Python app do something like this:
print("'(")
while allocating:
if random.random() > 0.5:
print(f"(allocate {random.randint(0, 20)})")
else:
print(f"(free {random.randint(0, 20)})")
...
print(")")
Giving you output looking like this (slightly prettyfied):
'((allocate 3)
(allocate 7)
(free 14)
(allocate 19)
...)
Remember what I said about quote ("tick") causing the default rule not to apply? Good. What would otherwise happen is that the values of allocate and free are looked up, and we don't want that. In our Lisp, we wish to do:
(dolist (entry allocation-log)
(case (first entry)
(allocate (plot-allocation (second entry)))
(free (plot-free (second entry)))))
For the data given above, the following sequence of function calls would have been made:
(plot-allocation 3)
(plot-allocation 7)
(plot-free 14)
(plot-allocation 19)
But What About list?
Well, sometimes you do want to evaluate the arguments. Say you have a nifty function manipulating a number and a string and returning a list of the resulting ... things. Let's make a false start:
(defun mess-with (number string)
'(value-of-number (1+ number) something-with-string (length string)))
Lisp> (mess-with 20 "foo")
(VALUE-OF-NUMBER (1+ NUMBER) SOMETHING-WITH-STRING (LENGTH STRING))
Hey! That's not what we wanted. We want to selectively evaluate some arguments, and leave the others as symbols. Try #2!
(defun mess-with (number string)
(list 'value-of-number (1+ number) 'something-with-string (length string)))
Lisp> (mess-with 20 "foo")
(VALUE-OF-NUMBER 21 SOMETHING-WITH-STRING 3)
Not Just quote, But backquote
Much better! Incidently, this pattern is so common in (mostly) macros, that there is special syntax for doing just that. The backquote:
(defun mess-with (number string)
`(value-of-number ,(1+ number) something-with-string ,(length string)))
It's like using quote, but with the option to explicitly evaluate some arguments by prefixing them with comma. The result is equivalent to using list, but if you're generating code from a macro you often only want to evaluate small parts of the code returned, so the backquote is more suited. For shorter lists, list can be more readable.
Hey, You Forgot About quote!
So, where does this leave us? Oh right, what does quote actually do? It simply returns its argument(s) unevaluated! Remember what I said in the beginning about regular functions? Turns out that some operators/functions need to not evaluate their arguments. Such as IF -- you wouldn't want the else branch to be evaluated if it wasn't taken, right? So-called special operators, together with macros, work like that. Special operators are also the "axiom" of the language -- minimal set of rules -- upon which you can implement the rest of Lisp by combining them together in different ways.
Back to quote, though:
Lisp> (quote spiffy-symbol)
SPIFFY-SYMBOL
Lisp> 'spiffy-symbol ; ' is just a shorthand ("reader macro"), as shown above
SPIFFY-SYMBOL
Compare to (on Steel-Bank Common Lisp):
Lisp> spiffy-symbol
debugger invoked on a UNBOUND-VARIABLE in thread #<THREAD "initial thread" RUNNING {A69F6A9}>:
The variable SPIFFY-SYMBOL is unbound.
Type HELP for debugger help, or (SB-EXT:QUIT) to exit from SBCL.
restarts (invokable by number or by possibly-abbreviated name):
0: [ABORT] Exit debugger, returning to top level.
(SB-INT:SIMPLE-EVAL-IN-LEXENV SPIFFY-SYMBOL #<NULL-LEXENV>)
0]
Because there is no spiffy-symbol in the current scope!
Summing Up
quote, backquote (with comma), and list are some of the tools you use to create lists, that are not only lists of values, but as you seen can be used as lightweight (no need to define a struct) data structures!
If you wish to learn more, I recommend Peter Seibel's book Practical Common Lisp for a practical approach to learning Lisp, if you're already into programming at large. Eventually on your Lisp journey, you'll start using packages too. Ron Garret's The Idiot's Guide to Common Lisp Packages will give you good explanation of those.
Happy hacking!
It says "don't evaluate me". For example, if you wanted to use a list as data, and not as code, you'd put a quote in front of it. For example,
(print '(+ 3 4)) prints "(+ 3 4)", whereas
(print (+ 3 4)) prints "7"
Other people have answered this question admirably, and Matthias Benkard brings up an excellent warning.
DO NOT USE QUOTE TO CREATE LISTS THAT YOU WILL LATER MODIFY. The spec allows the compiler to treat quoted lists as constants. Often, a compiler will optimize constants by creating a single value for them in memory and then referencing that single value from all locations where the constant appears. In other words, it may treat the constant like an anonymous global variable.
This can cause obvious problems. If you modify a constant, it may very well modify other uses of the same constant in completely unrelated code. For example, you may compare some variable to '(1 1) in some function, and in a completely different function, start a list with '(1 1) and then add more stuff to it. Upon running these functions, you may find that the first function doesn't match things properly anymore, because it's now trying to compare the variable to '(1 1 2 3 5 8 13), which is what the second function returned. These two functions are completely unrelated, but they have an effect on each other because of the use of constants. Even crazier bad effects can happen, like a perfectly normal list iteration suddenly infinite looping.
Use quote when you need a constant list, such as for comparison. Use list when you will be modifying the result.
One answer to this question says that QUOTE “creates list data structures”. This isn't quite right. QUOTE is more fundamental than this. In fact, QUOTE is a trivial operator: Its purpose is to prevent anything from happening at all. In particular, it doesn't create anything.
What (QUOTE X) says is basically “don't do anything, just give me X.” X needn't be a list as in (QUOTE (A B C)) or a symbol as in (QUOTE FOO). It can be any object whatever. Indeed, the result of evaluating the list that is produced by (LIST 'QUOTE SOME-OBJECT) will always just return SOME-OBJECT, whatever it is.
Now, the reason that (QUOTE (A B C)) seems as if it created a list whose elements are A, B, and C is that such a list really is what it returns; but at the time the QUOTE form is evaluated, the list has generally already been in existence for a while (as a component of the QUOTE form!), created either by the loader or the reader prior to execution of the code.
One implication of this that tends to trip up newbies fairly often is that it's very unwise to modify a list returned by a QUOTE form. Data returned by QUOTE is, for all intents and purposes, to be considered as part of the code being executed and should therefore be treated as read-only!
The quote prevents execution or evaluation of a form, turning it instead into data. In general you can execute the data by then eval'ing it.
quote creates list data structures, for example, the following are equivalent:
(quote a)
'a
It can also be used to create lists (or trees):
(quote (1 2 3))
'(1 2 3)
You're probably best off getting an introductary book on lisp, such as Practical Common Lisp (which is available to read on-line).
In Emacs Lisp:
What can be quoted ?
Lists and symbols.
Quoting a number evaluates to the number itself:
'5 is the same as 5.
What happens when you quote lists ?
For example:
'(one two) evaluates to
(list 'one 'two) which evaluates to
(list (intern "one") (intern ("two"))).
(intern "one") creates a symbol named "one" and stores it in a "central" hash-map, so anytime you say 'one then the symbol named "one" will be looked up in that central hash-map.
But what is a symbol ?
For example, in OO-languages (Java/Javascript/Python) a symbol could be represented as an object that has a name field, which is the symbol's name like "one" above, and data and/or code can be associated with it this object.
So an symbol in Python could be implemented as:
class Symbol:
def __init__(self,name,code,value):
self.name=name
self.code=code
self.value=value
In Emacs Lisp for example a symbol can have 1) data associated with it AND (at the same time - for the same symbol) 2) code associated with it - depending on the context, either the data or the code gets called.
For example, in Elisp:
(progn
(fset 'add '+ )
(set 'add 2)
(add add add)
)
evaluates to 4.
Because (add add add) evaluates as:
(add add add)
(+ add add)
(+ 2 add)
(+ 2 2)
4
So, for example, using the Symbol class we defined in Python above, this add ELisp-Symbol could be written in Python as Symbol("add",(lambda x,y: x+y),2).
Many thanks for folks on IRC #emacs for explaining symbols and quotes to me.
Code is data and data is code. There is no clear distinction between them.
This is a classical statement any lisp programmer knows.
When you quote a code, that code will be data.
1 ]=> '(+ 2 3 4)
;Value: (+ 2 3 4)
1 ]=> (+ 2 3 4)
;Value: 9
When you quote a code, the result will be data that represent that code. So, when you want to work with data that represents a program you quote that program. This is also valid for atomic expressions, not only for lists:
1 ]=> 'code
;Value: code
1 ]=> '10
;Value: 10
1 ]=> '"ok"
;Value: "ok"
1 ]=> code
;Unbound variable: code
Supposing you want to create a programming language embedded in lisp -- you will work with programs that are quoted in scheme (like '(+ 2 3)) and that are interpreted as code in the language you create, by giving programs a semantic interpretation. In this case you need to use quote to keep the data, otherwise it will be evaluated in external language.
When we want to pass an argument itself instead of passing the value of the argument then we use quote. It is mostly related to the procedure passing during using lists, pairs and atoms
which are not available in C programming Language ( most people start programming using C programming, Hence we get confused)
This is code in Scheme programming language which is a dialect of lisp and I guess you can understand this code.
(define atom? ; defining a procedure atom?
(lambda (x) ; which as one argument x
(and (not (null? x)) (not(pair? x) )))) ; checks if the argument is atom or not
(atom? '(a b c)) ; since it is a list it is false #f
The last line (atom? 'abc) is passing abc as it is to the procedure to check if abc is an atom or not, but when you pass(atom? abc) then it checks for the value of abc and passses the value to it. Since, we haven't provided any value to it
Quote returns the internal representation of its arguments. After plowing through way too many explanations of what quote doesn't do, that's when the light-bulb went on. If the REPL didn't convert function names to UPPER-CASE when I quoted them, it might not have dawned on me.
So. Ordinary Lisp functions convert their arguments into an internal representation, evaluate the arguments, and apply the function. Quote converts its arguments to an internal representation, and just returns that. Technically it's correct to say that quote says, "don't evaluate", but when I was trying to understand what it did, telling me what it doesn't do was frustrating. My toaster doesn't evaluate Lisp functions either; but that's not how you explain what a toaster does.
Anoter short answer:
quote means without evaluating it, and backquote is quote but leave back doors.
A good referrence:
Emacs Lisp Reference Manual make it very clear
9.3 Quoting
The special form quote returns its single argument, as written, without evaluating it. This provides a way to include constant symbols and lists, which are not self-evaluating objects, in a program. (It is not necessary to quote self-evaluating objects such as numbers, strings, and vectors.)
Special Form: quote object
This special form returns object, without evaluating it.
Because quote is used so often in programs, Lisp provides a convenient read syntax for it. An apostrophe character (‘'’) followed by a Lisp object (in read syntax) expands to a list whose first element is quote, and whose second element is the object. Thus, the read syntax 'x is an abbreviation for (quote x).
Here are some examples of expressions that use quote:
(quote (+ 1 2))
⇒ (+ 1 2)
(quote foo)
⇒ foo
'foo
⇒ foo
''foo
⇒ (quote foo)
'(quote foo)
⇒ (quote foo)
9.4 Backquote
Backquote constructs allow you to quote a list, but selectively evaluate elements of that list. In the simplest case, it is identical to the special form quote (described in the previous section; see Quoting). For example, these two forms yield identical results:
`(a list of (+ 2 3) elements)
⇒ (a list of (+ 2 3) elements)
'(a list of (+ 2 3) elements)
⇒ (a list of (+ 2 3) elements)
The special marker ‘,’ inside of the argument to backquote indicates a value that isn’t constant. The Emacs Lisp evaluator evaluates the argument of ‘,’, and puts the value in the list structure:
`(a list of ,(+ 2 3) elements)
⇒ (a list of 5 elements)
Substitution with ‘,’ is allowed at deeper levels of the list structure also. For example:
`(1 2 (3 ,(+ 4 5)))
⇒ (1 2 (3 9))
You can also splice an evaluated value into the resulting list, using the special marker ‘,#’. The elements of the spliced list become elements at the same level as the other elements of the resulting list. The equivalent code without using ‘`’ is often unreadable. Here are some examples:
(setq some-list '(2 3))
⇒ (2 3)
(cons 1 (append some-list '(4) some-list))
⇒ (1 2 3 4 2 3)
`(1 ,#some-list 4 ,#some-list)
⇒ (1 2 3 4 2 3)

How does `if` not evaluate all its arguments?

I'm trying to learn and understand the Lisp programming language to a deep level. The function + evaluates its arguments in applicative order:
(+ 1 (+ 1 2))
(+ 1 2) will be evaluated and then (+ 1 3) will be evaluated, but the if function works differently:
(if (> 1 2) (not-defined 1 2) 1)
As the form (not-defined 1 2) isn't evaluated, the program doesn't break.
How can the same syntax lead to different argument evaluation? How is the if function defined so that its arguments aren't evaluated?
if is a special operator, not an ordinary function.
This means that the normal rule that the rest elements in the compound form are evaluated before the function associated with the first element is invoked is not applicable (in that it is similar to macro forms).
The way this is implemented in a compiler and/or an interpreter is that one looks at the compound form and decides what to do with it based on its first element:
if it is a special operator, it does its special thing;
if it is a macro, its macro-function gets the whole form;
otherwise it is treated as a function - even if no function is defined.
Note that some special forms can be defined as macros expanding to other special forms, but some special forms must actually be present.
E.g., one can define if in terms of cond:
(defmacro my-if (condition yes no)
`(cond (,condition ,yes)
(t ,no)))
and vice versa (much more complicated - actually, cond is a macro, usually expanding into a sequence of ifs).
PS. Note that the distinction between system-supplied macros and special operators, while technically crisp and clear (see special-operator-p and macro-function), is ideologically blurred because
An implementation is free to implement a Common Lisp special operator
as a macro. An implementation is free to implement any macro operator
as a special operator, but only if an equivalent definition of the
macro is also provided.
sds's answer answers this question well, but there are a few more general aspects that I think are worth mentioning. As that answer and others have pointed out, if, is built into the language as a special operator, because it really is a kind of primitive. Most importantly, if is not a function.
That said, the functionality of if can be achieved using just functions and normal function calling where all the arguments are evaluated. Thus, conditionals can be implemented in the lambda calculus, on which languages in the family are somewhat based, but which doesn't have a conditional operator.
In the lambda calculus, one can define true and false as functions of two arguments. The arguments are presumed to be functions, and true calls the first of its arguments, and false calls the second. (This is a slight variation of Church booleans which simply return their first or second argument.)
true = λ[x y].(x)
false = λ[x y].(y)
(This is obviously a departure from boolean values in Common Lisp, where nil is false and anything else is true.) The benefit of this, though, is that we can use a boolean value to call one of two functions, depending on whether the boolean is true or false. Consider the Common Lisp form:
(if some-condition
then-part
else-part)
If were were using the booleans as defined above, then evaluating some-condition will produce either true or false, and if we were to call that result with the arguments
(lambda () then-part)
(lambda () else-part)
then only one of those would be called, so only one of then-part and else-part would actually be evaluated. In general, wrapping some forms up in a lambda is a good way to be able delay the evaluation of those forms.
The power of the Common Lisp macro system means that we could actually define an if macro using the types of booleans described above:
(defconstant true
(lambda (x y)
(declare (ignore y))
(funcall x)))
(defconstant false
(lambda (x y)
(declare (ignore x))
(funcall y)))
(defmacro new-if (test then &optional else)
`(funcall ,test
(lambda () ,then)
(lambda () ,else)))
With these definitions, some code like this:
(new-if (member 'a '(1 2 3))
(print "it's a member")
(print "it's not a member"))))
expands to this:
(FUNCALL (MEMBER 'A '(1 2 3)) ; assuming MEMBER were rewritten
(LAMBDA () (PRINT "it's a member")) ; to return `true` or `false`
(LAMBDA () (PRINT "it's not a member")))
In general, if there is some form and some of the arguments aren't getting evaluated, then the (car of the) form is either a Common Lisp special operator or a macro. If you need to write a function where the arguments will be evaluated, but you want some forms not to be evaluated, you can wrap them up in lambda expressions and have your function call those anonymous functions conditionally.
This is a possible way to implement if, if you didn't already have it in the language. Of course, modern computer hardware isn't based on a lambda calculus interpreter, but rather on CPUs that have test and jump instructions, so it's more efficient for the language to provide if a primitive and to compile down to the appropriate machine instructions.
Lisp syntax is regular, much more regular than other languages, but it's still not completely regular: for example in
(let ((x 0))
x)
let is not the name of a function and ((x 0)) is not a bad form in which a list that is not a lambda form has been used in the first position.
There are quite a few "special cases" (still a lot less than other languages, of course) where the general rule of each list being a function call is not followed, and if is one of them. Common Lisp has quite a few "special forms" (because absolute minimality was not the point) but you can get away for example in a scheme dialect with just five of them: if, progn, quote, lambda and set! (or six if you want macros).
While the syntax of Lisp is not totally uniform the underlying representation of code is however quite uniform (just lists and atoms) and the uniformity and simplicity of representation is what facilitates metaprogramming (macros).
"Lisp has no syntax" is a statement with some truth in it, but so it's the statement "Lisp has two syntaxes": one syntax is what uses the reader to convert from character streams to s-expressions, another syntax is what uses the compiler/evaluator to convert from s-expressions to executable code.
It's also true that Lisp has no syntax because neither of those two levels is fixed. Differently from other programming languages you can customize both the first step (using reader macros) and the second step (using macros).
It would not make any sense to do so. Example: (if (ask-user-should-i-quit) (quit) (continue)). Should that quit, even though the user does not want to?
IF is not a function in Lisp. It is a special built-in operator. Lisp a several built-in special operators. See: Special Forms. Those are not functions.
The arguments are not evaluated as for functions, because if is a special operator. Special operators can be evaluated in any arbitrary way, that's why they're called special.
Consider e.g.
(if (not (= x 0))
(/ y x))
If the division was always evaluated, there could be a division by zero error which obviously was not intended.
If isn't a function, it's a special form. If you wanted to implement similar functionality yourself, you could do so by defining a macro rather than a function.
This answer applies to Common Lisp, but it'll probably the same for most other Lisps (though in some if may be a macro rather than a special form).

Confused by Lisp Quoting

I have a question concerning evaluation of lists in lisp.
Why is (a) and (+ a 1) not evaluated,
(defun test (a) (+ a 1))
just like (print 4) is not evaluated here
(if (< 1 2) (print 3) (print 4))
but (print (+ 2 3)) is evaluated here
(test (print (+ 2 3)))
Does it have something to do with them being standard library functions? Is it possible for me to define functions like that in my lisp program?
As you probably know, Lisp compound forms are generally processed from the outside in. You must look at the symbol in the first position of the outermost nesting to understand a form. That symbol completely determines the meaning of the form. The following expressions all contain (b c) with completely different meaning; therefore, we cannot understand them by analyzing the (b c) part first:
;; Common Lisp: define a class A derived from B and C
(defclass a (b c) ())
;; Common Lisp: define a function of two arguments
(defun a (b c) ())
;; add A to the result of calling function B on variable C:
(+ a (b c))
Traditionally, Lisp dialects have divided forms into operator forms and function call forms. An operator form has a completely arbitrary meaning, determined by the piece of code which compiles or interprets that functions (e.g. the evaluation simply recurses over all of the function call's argument forms, and the resulting values are passed to the function).
From the early history, Lisp has allowed users to write their own operators. There existed two approaches to this: interpretive operators (historically known as fexprs) and compiling operators known as macros. Both hinge around the idea of a function which receives the unevaluated form as an argument, so that it can implement a custom strategy, thereby extending the evaluation model with new behaviors.
A fexpr type operator is simply handed the form at run-time, along with an environment object with which it can look up the values of variables and such. That operator then walks the form and implements the behavior.
A macro operator is handed the form at macro-expansion time (which usually happens when top-level forms are read, just before they are evaluated or compiled). Its job is not to interpret the form's behavior, but instead to translate it by generating code. I.e. a macro is a mini compiler. (Generated code can contain more macro calls; the macro expander will take care of that, ensuring that all macro calls are decimated.)
The fexpr approach fell out of favor, most likely because it is inefficient. It basically makes compilation impossible, whereas Lisp hackers valued compilation. (Lisp was already a compiled language from as early as circa 1960.) The fexpr approach is also hostile toward lexical environments; it requires the fexpr, which is a function, to be able to peer into the variable binding environment of the form in which its invoked, which is a kind of encapsulation violation that is not allowed by lexical scopes.
Macro writing is slightly more difficult, and in some ways less flexible than fexprs, but support for macro writing improved in Lisp through the 1960's into the 70's to make it close to as easy as possible. Macro originally had receive the whole form and then have to parse it themselves. The macro-defining system developed into something that provides macro functions with arguments that receive the broken-down syntax in easily digestible pieces, including some nested aspects of the syntax. The backquote syntax for writing code templates was also developed, making it much easier to express code generation.
So to answer your question, how can I write forms like that myself? For instance if:
;; Imitation of old-fashioned technique: receive the whole form,
;; extract parts from it and return the translation.
;; Common Lisp defmacro supports this via the &whole keyword
;; in macro lambda lists which lets us have access to the whole form.
;;
;; (Because we are using defmacro, we need to declare arguments "an co &optional al",
;; to make this a three argument macro with an optional third argument, but
;; we don't use those arguments. In ancient lisps, they would not appear:
;; a macro would be a one-argument function, and would have to check the number
;; of arguments itself, to flag bad syntax like (my-if 42) or (my-if).)
;;
(defmacro my-if (&whole if-form an co &optional al)
(let ((antecedent (second if-form)) ;; extract pieces ourselves
(consequent (third if-form)) ;; from whole (my-if ...) form
(alternative (fourth if-form)))
(list 'cond (list antecedent consequent) (list t alternative))))
;; "Modern" version. Use the parsed arguments, and also take advantage of
;; backquote syntax to write the COND with a syntax that looks like the code.
(defmacro my-if (antecedent consequent &optional alternative)
`(cond (,antecedent ,consequent) (t ,alternative))))
This is a fitting example because originally Lisp only had cond. There was no if in McCarthy's Lisp. That "syntactic sugar" was invented later, probably as a macro expanding to cond, just like my-if above.
if and defun are macros. Macros expand a form into a longer piece of code. At expansion time, none of the macro's arguments are evaluated.
When you try to write a function, but struggle because you need to implement a custom evaluation strategy, its a strong signal that you should be writing a macro instead.
Disclaimer: Depending on what kind of lisp you are using, if and defun might technically be called "special forms" and not macros, but the concept of delayed evaluation still applies.
Lisp consists of a model of evaluation of forms. Different Lisp dialects have different rules for those.
Let's look at Common Lisp.
data evaluates to itself
a function form is evaluated by calling the function on the evaluated arguments
special forms are evaluated according to rules defined for each special operator. The Common Lisp standard lists all of those, defines what they do in an informal way and there is no way to define new special operators by the user.
macros forms are transformed, the result is evaluated
How IF, DEFUN etc. works and what they evaluated, when it is doen and what is not evaluated is defined in the Common Lisp standard.

What is the difference between 1 and '1 in Lisp?

I had never really thought about whether a symbol could be a number in Lisp, so I played around with it today:
> '1
1
> (+ '1 '1)
2
> (+ '1 1)
2
> (define a '1)
> (+ a 1)
2
The above code is scheme, but it seems to be roughly the same in Common Lisp and Clojure as well. Is there any difference between 1 and quoted 1?
In Common Lisp, '1 is shorthand for (QUOTE 1). When evaluated, (QUOTE something) returns the something part, unevaluated. However, there is no difference between 1 evaluated and 1 unevaluated.
So there is a difference to the reader: '1 reads as (QUOTE 1) and 1 reads as 1. But there is no difference when evaluted.
Numbers are self-evaluating objects. That's why you don't have to worry about quoting them, as you do with, say, lists.
A symbol can be made from any string. If you want the symbol whose name is the single character 1, you can say:
(intern "1")
which prints |1|, suggesting an alternate way to enter it:
'|1|
Quoting prevents expressions from being evaluated until later. For example, the following is not a proper list:
(1 2 3)
This is because Lisp interprets 1 as a function, which it is not. So the list must be quoted:
'(1 2 3)
When you quote a very simple expression such as a number, Lisp effectively does not alter its behavior.
See Wikipedia: Lisp.
Well, they are in fact very different. '1 is however precisely the same as (quote 1). (car ''x) evaluates to the symbol 'quote'.
1 is an S-expression, it's the external representation of a datum, a number 1. To say that 1 is a 'number-object' or an S-expression to enter that object would both be acceptable. Often it is said that 1 is the external representation for the actual number object.
(quote 1) is another S-expression, it's an S-expression for a list whose first element is the symbol 'quote' and whose second element is the number 1. This is where it's already different, syntactic keywords, unlike functions, are not considered objects in the language and they do not evaluate to them.
However, both are external representations of objects (data) which evaluate to the same datum. The number whose external representation is 1, they are however most certainly not the same objects, the same, code, the same datum the same whatever, they just evaluate to the very same thing. Numbers evaluate to themselves. To say that they are the same is to say that:
(+ 1 (* 3 3))
And
(if "Strings are true" (* 5 (- 5 3)) "Strings are not true? This must be a bug!")
Are 'the same', they aren't, they are both different programs which just happen to terminate to the same value, a lisp form is also a program, a form is a datum which is also a program, remember.
Also, I was taught a handy trick once that shows that self-evaluating data are truly not symbols when entered:
(let ((num 4))
(symbol? num) ; ====> evaluates to #f
(symbol? 'num) ; ====> evaluates to #t
(symbol? '4) ; ====> evaluates to #f
(symbol? '#\c) ; #f again, et cetera
(symbol? (car ''x)) ; #t
(symbol? quote) ; error, in most implementations
)
Self evaluating data truly evaluate to themselves, they are not 'predefined symbols' of some sorts.
In Lisp, the apostrophe prevents symbols to be evaluated. Using an apostrophe before a number is not forbidden, it is not necessary as the numbers represent themselves. However, like any other list, it automatically gets transformed to an appropriate function call. The interpreter considers these numbers coincide with their value.
As has been pointed out, there is no difference, as numbers evaluate to themselves. You can confirm this by using eval:
(eval 1) ;=> 1
This is not limited to numbers, by the way. In fact, in Common Lisp, most things evaluate to themselves. It's just that it's very rare for something other than numbers, strings, symbols, and lists to be evaluated. For instance, the following works:
(eval (make-hash-table)) ;equivalent to just (make-hash-table)
In Lisp, quote prevent the following expression to be evaluated. ' is a shorthand for quote. As a result, '1 is same as (quote 1).
However, in Lisp, symbols can never be a number. I mean, 'abc is a symbol, but '123 is not (evaluated into) a symbol. I think this is wrong of the design of Lisp. Another case is not only #t or #f can be used as a Boolean expression.

When to use ' (or quote) in Lisp?

After making it through the major parts of an introductory Lisp book, I still couldn't understand what the special operator (quote) (or equivalent ') function does, yet this has been all over Lisp code that I've seen.
What does it do?
Short answer
Bypass the default evaluation rules and do not evaluate the expression (symbol or s-exp), passing it along to the function exactly as typed.
Long Answer: The Default Evaluation Rule
When a regular (I'll come to that later) function is invoked, all arguments passed to it are evaluated. This means you can write this:
(* (+ a 2)
3)
Which in turn evaluates (+ a 2), by evaluating a and 2. The value of the symbol a is looked up in the current variable binding set, and then replaced. Say a is currently bound to the value 3:
(let ((a 3))
(* (+ a 2)
3))
We'd get (+ 3 2), + is then invoked on 3 and 2 yielding 5. Our original form is now (* 5 3) yielding 15.
Explain quote Already!
Alright. As seen above, all arguments to a function are evaluated, so if you would like to pass the symbol a and not its value, you don't want to evaluate it. Lisp symbols can double both as their values, and markers where you in other languages would have used strings, such as keys to hash tables.
This is where quote comes in. Say you want to plot resource allocations from a Python application, but rather do the plotting in Lisp. Have your Python app do something like this:
print("'(")
while allocating:
if random.random() > 0.5:
print(f"(allocate {random.randint(0, 20)})")
else:
print(f"(free {random.randint(0, 20)})")
...
print(")")
Giving you output looking like this (slightly prettyfied):
'((allocate 3)
(allocate 7)
(free 14)
(allocate 19)
...)
Remember what I said about quote ("tick") causing the default rule not to apply? Good. What would otherwise happen is that the values of allocate and free are looked up, and we don't want that. In our Lisp, we wish to do:
(dolist (entry allocation-log)
(case (first entry)
(allocate (plot-allocation (second entry)))
(free (plot-free (second entry)))))
For the data given above, the following sequence of function calls would have been made:
(plot-allocation 3)
(plot-allocation 7)
(plot-free 14)
(plot-allocation 19)
But What About list?
Well, sometimes you do want to evaluate the arguments. Say you have a nifty function manipulating a number and a string and returning a list of the resulting ... things. Let's make a false start:
(defun mess-with (number string)
'(value-of-number (1+ number) something-with-string (length string)))
Lisp> (mess-with 20 "foo")
(VALUE-OF-NUMBER (1+ NUMBER) SOMETHING-WITH-STRING (LENGTH STRING))
Hey! That's not what we wanted. We want to selectively evaluate some arguments, and leave the others as symbols. Try #2!
(defun mess-with (number string)
(list 'value-of-number (1+ number) 'something-with-string (length string)))
Lisp> (mess-with 20 "foo")
(VALUE-OF-NUMBER 21 SOMETHING-WITH-STRING 3)
Not Just quote, But backquote
Much better! Incidently, this pattern is so common in (mostly) macros, that there is special syntax for doing just that. The backquote:
(defun mess-with (number string)
`(value-of-number ,(1+ number) something-with-string ,(length string)))
It's like using quote, but with the option to explicitly evaluate some arguments by prefixing them with comma. The result is equivalent to using list, but if you're generating code from a macro you often only want to evaluate small parts of the code returned, so the backquote is more suited. For shorter lists, list can be more readable.
Hey, You Forgot About quote!
So, where does this leave us? Oh right, what does quote actually do? It simply returns its argument(s) unevaluated! Remember what I said in the beginning about regular functions? Turns out that some operators/functions need to not evaluate their arguments. Such as IF -- you wouldn't want the else branch to be evaluated if it wasn't taken, right? So-called special operators, together with macros, work like that. Special operators are also the "axiom" of the language -- minimal set of rules -- upon which you can implement the rest of Lisp by combining them together in different ways.
Back to quote, though:
Lisp> (quote spiffy-symbol)
SPIFFY-SYMBOL
Lisp> 'spiffy-symbol ; ' is just a shorthand ("reader macro"), as shown above
SPIFFY-SYMBOL
Compare to (on Steel-Bank Common Lisp):
Lisp> spiffy-symbol
debugger invoked on a UNBOUND-VARIABLE in thread #<THREAD "initial thread" RUNNING {A69F6A9}>:
The variable SPIFFY-SYMBOL is unbound.
Type HELP for debugger help, or (SB-EXT:QUIT) to exit from SBCL.
restarts (invokable by number or by possibly-abbreviated name):
0: [ABORT] Exit debugger, returning to top level.
(SB-INT:SIMPLE-EVAL-IN-LEXENV SPIFFY-SYMBOL #<NULL-LEXENV>)
0]
Because there is no spiffy-symbol in the current scope!
Summing Up
quote, backquote (with comma), and list are some of the tools you use to create lists, that are not only lists of values, but as you seen can be used as lightweight (no need to define a struct) data structures!
If you wish to learn more, I recommend Peter Seibel's book Practical Common Lisp for a practical approach to learning Lisp, if you're already into programming at large. Eventually on your Lisp journey, you'll start using packages too. Ron Garret's The Idiot's Guide to Common Lisp Packages will give you good explanation of those.
Happy hacking!
It says "don't evaluate me". For example, if you wanted to use a list as data, and not as code, you'd put a quote in front of it. For example,
(print '(+ 3 4)) prints "(+ 3 4)", whereas
(print (+ 3 4)) prints "7"
Other people have answered this question admirably, and Matthias Benkard brings up an excellent warning.
DO NOT USE QUOTE TO CREATE LISTS THAT YOU WILL LATER MODIFY. The spec allows the compiler to treat quoted lists as constants. Often, a compiler will optimize constants by creating a single value for them in memory and then referencing that single value from all locations where the constant appears. In other words, it may treat the constant like an anonymous global variable.
This can cause obvious problems. If you modify a constant, it may very well modify other uses of the same constant in completely unrelated code. For example, you may compare some variable to '(1 1) in some function, and in a completely different function, start a list with '(1 1) and then add more stuff to it. Upon running these functions, you may find that the first function doesn't match things properly anymore, because it's now trying to compare the variable to '(1 1 2 3 5 8 13), which is what the second function returned. These two functions are completely unrelated, but they have an effect on each other because of the use of constants. Even crazier bad effects can happen, like a perfectly normal list iteration suddenly infinite looping.
Use quote when you need a constant list, such as for comparison. Use list when you will be modifying the result.
One answer to this question says that QUOTE “creates list data structures”. This isn't quite right. QUOTE is more fundamental than this. In fact, QUOTE is a trivial operator: Its purpose is to prevent anything from happening at all. In particular, it doesn't create anything.
What (QUOTE X) says is basically “don't do anything, just give me X.” X needn't be a list as in (QUOTE (A B C)) or a symbol as in (QUOTE FOO). It can be any object whatever. Indeed, the result of evaluating the list that is produced by (LIST 'QUOTE SOME-OBJECT) will always just return SOME-OBJECT, whatever it is.
Now, the reason that (QUOTE (A B C)) seems as if it created a list whose elements are A, B, and C is that such a list really is what it returns; but at the time the QUOTE form is evaluated, the list has generally already been in existence for a while (as a component of the QUOTE form!), created either by the loader or the reader prior to execution of the code.
One implication of this that tends to trip up newbies fairly often is that it's very unwise to modify a list returned by a QUOTE form. Data returned by QUOTE is, for all intents and purposes, to be considered as part of the code being executed and should therefore be treated as read-only!
The quote prevents execution or evaluation of a form, turning it instead into data. In general you can execute the data by then eval'ing it.
quote creates list data structures, for example, the following are equivalent:
(quote a)
'a
It can also be used to create lists (or trees):
(quote (1 2 3))
'(1 2 3)
You're probably best off getting an introductary book on lisp, such as Practical Common Lisp (which is available to read on-line).
In Emacs Lisp:
What can be quoted ?
Lists and symbols.
Quoting a number evaluates to the number itself:
'5 is the same as 5.
What happens when you quote lists ?
For example:
'(one two) evaluates to
(list 'one 'two) which evaluates to
(list (intern "one") (intern ("two"))).
(intern "one") creates a symbol named "one" and stores it in a "central" hash-map, so anytime you say 'one then the symbol named "one" will be looked up in that central hash-map.
But what is a symbol ?
For example, in OO-languages (Java/Javascript/Python) a symbol could be represented as an object that has a name field, which is the symbol's name like "one" above, and data and/or code can be associated with it this object.
So an symbol in Python could be implemented as:
class Symbol:
def __init__(self,name,code,value):
self.name=name
self.code=code
self.value=value
In Emacs Lisp for example a symbol can have 1) data associated with it AND (at the same time - for the same symbol) 2) code associated with it - depending on the context, either the data or the code gets called.
For example, in Elisp:
(progn
(fset 'add '+ )
(set 'add 2)
(add add add)
)
evaluates to 4.
Because (add add add) evaluates as:
(add add add)
(+ add add)
(+ 2 add)
(+ 2 2)
4
So, for example, using the Symbol class we defined in Python above, this add ELisp-Symbol could be written in Python as Symbol("add",(lambda x,y: x+y),2).
Many thanks for folks on IRC #emacs for explaining symbols and quotes to me.
Code is data and data is code. There is no clear distinction between them.
This is a classical statement any lisp programmer knows.
When you quote a code, that code will be data.
1 ]=> '(+ 2 3 4)
;Value: (+ 2 3 4)
1 ]=> (+ 2 3 4)
;Value: 9
When you quote a code, the result will be data that represent that code. So, when you want to work with data that represents a program you quote that program. This is also valid for atomic expressions, not only for lists:
1 ]=> 'code
;Value: code
1 ]=> '10
;Value: 10
1 ]=> '"ok"
;Value: "ok"
1 ]=> code
;Unbound variable: code
Supposing you want to create a programming language embedded in lisp -- you will work with programs that are quoted in scheme (like '(+ 2 3)) and that are interpreted as code in the language you create, by giving programs a semantic interpretation. In this case you need to use quote to keep the data, otherwise it will be evaluated in external language.
When we want to pass an argument itself instead of passing the value of the argument then we use quote. It is mostly related to the procedure passing during using lists, pairs and atoms
which are not available in C programming Language ( most people start programming using C programming, Hence we get confused)
This is code in Scheme programming language which is a dialect of lisp and I guess you can understand this code.
(define atom? ; defining a procedure atom?
(lambda (x) ; which as one argument x
(and (not (null? x)) (not(pair? x) )))) ; checks if the argument is atom or not
(atom? '(a b c)) ; since it is a list it is false #f
The last line (atom? 'abc) is passing abc as it is to the procedure to check if abc is an atom or not, but when you pass(atom? abc) then it checks for the value of abc and passses the value to it. Since, we haven't provided any value to it
Quote returns the internal representation of its arguments. After plowing through way too many explanations of what quote doesn't do, that's when the light-bulb went on. If the REPL didn't convert function names to UPPER-CASE when I quoted them, it might not have dawned on me.
So. Ordinary Lisp functions convert their arguments into an internal representation, evaluate the arguments, and apply the function. Quote converts its arguments to an internal representation, and just returns that. Technically it's correct to say that quote says, "don't evaluate", but when I was trying to understand what it did, telling me what it doesn't do was frustrating. My toaster doesn't evaluate Lisp functions either; but that's not how you explain what a toaster does.
Anoter short answer:
quote means without evaluating it, and backquote is quote but leave back doors.
A good referrence:
Emacs Lisp Reference Manual make it very clear
9.3 Quoting
The special form quote returns its single argument, as written, without evaluating it. This provides a way to include constant symbols and lists, which are not self-evaluating objects, in a program. (It is not necessary to quote self-evaluating objects such as numbers, strings, and vectors.)
Special Form: quote object
This special form returns object, without evaluating it.
Because quote is used so often in programs, Lisp provides a convenient read syntax for it. An apostrophe character (‘'’) followed by a Lisp object (in read syntax) expands to a list whose first element is quote, and whose second element is the object. Thus, the read syntax 'x is an abbreviation for (quote x).
Here are some examples of expressions that use quote:
(quote (+ 1 2))
⇒ (+ 1 2)
(quote foo)
⇒ foo
'foo
⇒ foo
''foo
⇒ (quote foo)
'(quote foo)
⇒ (quote foo)
9.4 Backquote
Backquote constructs allow you to quote a list, but selectively evaluate elements of that list. In the simplest case, it is identical to the special form quote (described in the previous section; see Quoting). For example, these two forms yield identical results:
`(a list of (+ 2 3) elements)
⇒ (a list of (+ 2 3) elements)
'(a list of (+ 2 3) elements)
⇒ (a list of (+ 2 3) elements)
The special marker ‘,’ inside of the argument to backquote indicates a value that isn’t constant. The Emacs Lisp evaluator evaluates the argument of ‘,’, and puts the value in the list structure:
`(a list of ,(+ 2 3) elements)
⇒ (a list of 5 elements)
Substitution with ‘,’ is allowed at deeper levels of the list structure also. For example:
`(1 2 (3 ,(+ 4 5)))
⇒ (1 2 (3 9))
You can also splice an evaluated value into the resulting list, using the special marker ‘,#’. The elements of the spliced list become elements at the same level as the other elements of the resulting list. The equivalent code without using ‘`’ is often unreadable. Here are some examples:
(setq some-list '(2 3))
⇒ (2 3)
(cons 1 (append some-list '(4) some-list))
⇒ (1 2 3 4 2 3)
`(1 ,#some-list 4 ,#some-list)
⇒ (1 2 3 4 2 3)