Generalized Threading Macro in Clojure - macros

Note: this is NOT about concurrency. This is about the thread macro.
I know that -> puts the object at the 2nd position and ->> puts the argument at the last position.
Now, I'm curious, much like the short hand notation of #( ... % ) for functions, is there a short hand notation for threads that lets me place the argument at arbitrary location?
The goal would be that instead of having a fixed location for the thread to run through ... I can write arbitrary forms, and insert %% at special places, and the %% is where the thread gets inserted.
Thanks!

There is now a generalized threading macro in Clojure since 1.5 called as->.
This tweet gives an example of how it works: https://twitter.com/borkdude/status/302881431649128448
(as-> "/tmp" x
(java.io.File. x)
(file-seq x)
(filter (memfn isDirectory) x)
(count x))
First 'x' is bound to "/tmp" and a file is made out of it. 'x' is rebound again to the resulting file and a put through the 'file-seq' function, etc.

The 'diamond wand' from Swiss Arrows library would do what you're asking for:
(-<> 0
(* <> 5)
(vector 1 2 <> 3 4))
; => [1 2 0 3 4]
That said, it isn't something you end up needing often (or ever in my Clojure experience)

In case anyone else comes across this, there is a reason the provided macros exist, but an arbitrary placement one does not: the latter would lead to poor API design.
The -> macro places the argument in the first position. This corresponds to functions that work on some subject argument, e.g., conj, assoc.
The ->> macro places the argument in the last position. This corresponds to functions that work on sequences, e.g., map, reduce.
Design your APIs well, and you'll be less likely to need such a macro.

There was a library that provided this feature, but I forgot where. It might of been in the deprecated clojure-contrib. It was the -$> macro.
But you could derive one from clojure's core -> macro to make the one you're looking for:
(defmacro -$>
([x] x)
([x form] (if (seq? form)
(with-meta (map #(if (= %1 '$) x %1) form) (meta form))
(list form x)))
([x form & more] `(-$> (-$> ~x ~form) ~#more)))
And use $ to indicate the insertion point:
user=> (-$> 2 str (identity $) (println $))
2
nil
Technically, you could use multiple $ in one form. But this implementation suffers from expanding the same form multiple times (in exchange for simplicity).

Related

LISP: when modifying a list (with `nth`), all elements change [duplicate]

After making it through the major parts of an introductory Lisp book, I still couldn't understand what the special operator (quote) (or equivalent ') function does, yet this has been all over Lisp code that I've seen.
What does it do?
Short answer
Bypass the default evaluation rules and do not evaluate the expression (symbol or s-exp), passing it along to the function exactly as typed.
Long Answer: The Default Evaluation Rule
When a regular (I'll come to that later) function is invoked, all arguments passed to it are evaluated. This means you can write this:
(* (+ a 2)
3)
Which in turn evaluates (+ a 2), by evaluating a and 2. The value of the symbol a is looked up in the current variable binding set, and then replaced. Say a is currently bound to the value 3:
(let ((a 3))
(* (+ a 2)
3))
We'd get (+ 3 2), + is then invoked on 3 and 2 yielding 5. Our original form is now (* 5 3) yielding 15.
Explain quote Already!
Alright. As seen above, all arguments to a function are evaluated, so if you would like to pass the symbol a and not its value, you don't want to evaluate it. Lisp symbols can double both as their values, and markers where you in other languages would have used strings, such as keys to hash tables.
This is where quote comes in. Say you want to plot resource allocations from a Python application, but rather do the plotting in Lisp. Have your Python app do something like this:
print("'(")
while allocating:
if random.random() > 0.5:
print(f"(allocate {random.randint(0, 20)})")
else:
print(f"(free {random.randint(0, 20)})")
...
print(")")
Giving you output looking like this (slightly prettyfied):
'((allocate 3)
(allocate 7)
(free 14)
(allocate 19)
...)
Remember what I said about quote ("tick") causing the default rule not to apply? Good. What would otherwise happen is that the values of allocate and free are looked up, and we don't want that. In our Lisp, we wish to do:
(dolist (entry allocation-log)
(case (first entry)
(allocate (plot-allocation (second entry)))
(free (plot-free (second entry)))))
For the data given above, the following sequence of function calls would have been made:
(plot-allocation 3)
(plot-allocation 7)
(plot-free 14)
(plot-allocation 19)
But What About list?
Well, sometimes you do want to evaluate the arguments. Say you have a nifty function manipulating a number and a string and returning a list of the resulting ... things. Let's make a false start:
(defun mess-with (number string)
'(value-of-number (1+ number) something-with-string (length string)))
Lisp> (mess-with 20 "foo")
(VALUE-OF-NUMBER (1+ NUMBER) SOMETHING-WITH-STRING (LENGTH STRING))
Hey! That's not what we wanted. We want to selectively evaluate some arguments, and leave the others as symbols. Try #2!
(defun mess-with (number string)
(list 'value-of-number (1+ number) 'something-with-string (length string)))
Lisp> (mess-with 20 "foo")
(VALUE-OF-NUMBER 21 SOMETHING-WITH-STRING 3)
Not Just quote, But backquote
Much better! Incidently, this pattern is so common in (mostly) macros, that there is special syntax for doing just that. The backquote:
(defun mess-with (number string)
`(value-of-number ,(1+ number) something-with-string ,(length string)))
It's like using quote, but with the option to explicitly evaluate some arguments by prefixing them with comma. The result is equivalent to using list, but if you're generating code from a macro you often only want to evaluate small parts of the code returned, so the backquote is more suited. For shorter lists, list can be more readable.
Hey, You Forgot About quote!
So, where does this leave us? Oh right, what does quote actually do? It simply returns its argument(s) unevaluated! Remember what I said in the beginning about regular functions? Turns out that some operators/functions need to not evaluate their arguments. Such as IF -- you wouldn't want the else branch to be evaluated if it wasn't taken, right? So-called special operators, together with macros, work like that. Special operators are also the "axiom" of the language -- minimal set of rules -- upon which you can implement the rest of Lisp by combining them together in different ways.
Back to quote, though:
Lisp> (quote spiffy-symbol)
SPIFFY-SYMBOL
Lisp> 'spiffy-symbol ; ' is just a shorthand ("reader macro"), as shown above
SPIFFY-SYMBOL
Compare to (on Steel-Bank Common Lisp):
Lisp> spiffy-symbol
debugger invoked on a UNBOUND-VARIABLE in thread #<THREAD "initial thread" RUNNING {A69F6A9}>:
The variable SPIFFY-SYMBOL is unbound.
Type HELP for debugger help, or (SB-EXT:QUIT) to exit from SBCL.
restarts (invokable by number or by possibly-abbreviated name):
0: [ABORT] Exit debugger, returning to top level.
(SB-INT:SIMPLE-EVAL-IN-LEXENV SPIFFY-SYMBOL #<NULL-LEXENV>)
0]
Because there is no spiffy-symbol in the current scope!
Summing Up
quote, backquote (with comma), and list are some of the tools you use to create lists, that are not only lists of values, but as you seen can be used as lightweight (no need to define a struct) data structures!
If you wish to learn more, I recommend Peter Seibel's book Practical Common Lisp for a practical approach to learning Lisp, if you're already into programming at large. Eventually on your Lisp journey, you'll start using packages too. Ron Garret's The Idiot's Guide to Common Lisp Packages will give you good explanation of those.
Happy hacking!
It says "don't evaluate me". For example, if you wanted to use a list as data, and not as code, you'd put a quote in front of it. For example,
(print '(+ 3 4)) prints "(+ 3 4)", whereas
(print (+ 3 4)) prints "7"
Other people have answered this question admirably, and Matthias Benkard brings up an excellent warning.
DO NOT USE QUOTE TO CREATE LISTS THAT YOU WILL LATER MODIFY. The spec allows the compiler to treat quoted lists as constants. Often, a compiler will optimize constants by creating a single value for them in memory and then referencing that single value from all locations where the constant appears. In other words, it may treat the constant like an anonymous global variable.
This can cause obvious problems. If you modify a constant, it may very well modify other uses of the same constant in completely unrelated code. For example, you may compare some variable to '(1 1) in some function, and in a completely different function, start a list with '(1 1) and then add more stuff to it. Upon running these functions, you may find that the first function doesn't match things properly anymore, because it's now trying to compare the variable to '(1 1 2 3 5 8 13), which is what the second function returned. These two functions are completely unrelated, but they have an effect on each other because of the use of constants. Even crazier bad effects can happen, like a perfectly normal list iteration suddenly infinite looping.
Use quote when you need a constant list, such as for comparison. Use list when you will be modifying the result.
One answer to this question says that QUOTE “creates list data structures”. This isn't quite right. QUOTE is more fundamental than this. In fact, QUOTE is a trivial operator: Its purpose is to prevent anything from happening at all. In particular, it doesn't create anything.
What (QUOTE X) says is basically “don't do anything, just give me X.” X needn't be a list as in (QUOTE (A B C)) or a symbol as in (QUOTE FOO). It can be any object whatever. Indeed, the result of evaluating the list that is produced by (LIST 'QUOTE SOME-OBJECT) will always just return SOME-OBJECT, whatever it is.
Now, the reason that (QUOTE (A B C)) seems as if it created a list whose elements are A, B, and C is that such a list really is what it returns; but at the time the QUOTE form is evaluated, the list has generally already been in existence for a while (as a component of the QUOTE form!), created either by the loader or the reader prior to execution of the code.
One implication of this that tends to trip up newbies fairly often is that it's very unwise to modify a list returned by a QUOTE form. Data returned by QUOTE is, for all intents and purposes, to be considered as part of the code being executed and should therefore be treated as read-only!
The quote prevents execution or evaluation of a form, turning it instead into data. In general you can execute the data by then eval'ing it.
quote creates list data structures, for example, the following are equivalent:
(quote a)
'a
It can also be used to create lists (or trees):
(quote (1 2 3))
'(1 2 3)
You're probably best off getting an introductary book on lisp, such as Practical Common Lisp (which is available to read on-line).
In Emacs Lisp:
What can be quoted ?
Lists and symbols.
Quoting a number evaluates to the number itself:
'5 is the same as 5.
What happens when you quote lists ?
For example:
'(one two) evaluates to
(list 'one 'two) which evaluates to
(list (intern "one") (intern ("two"))).
(intern "one") creates a symbol named "one" and stores it in a "central" hash-map, so anytime you say 'one then the symbol named "one" will be looked up in that central hash-map.
But what is a symbol ?
For example, in OO-languages (Java/Javascript/Python) a symbol could be represented as an object that has a name field, which is the symbol's name like "one" above, and data and/or code can be associated with it this object.
So an symbol in Python could be implemented as:
class Symbol:
def __init__(self,name,code,value):
self.name=name
self.code=code
self.value=value
In Emacs Lisp for example a symbol can have 1) data associated with it AND (at the same time - for the same symbol) 2) code associated with it - depending on the context, either the data or the code gets called.
For example, in Elisp:
(progn
(fset 'add '+ )
(set 'add 2)
(add add add)
)
evaluates to 4.
Because (add add add) evaluates as:
(add add add)
(+ add add)
(+ 2 add)
(+ 2 2)
4
So, for example, using the Symbol class we defined in Python above, this add ELisp-Symbol could be written in Python as Symbol("add",(lambda x,y: x+y),2).
Many thanks for folks on IRC #emacs for explaining symbols and quotes to me.
Code is data and data is code. There is no clear distinction between them.
This is a classical statement any lisp programmer knows.
When you quote a code, that code will be data.
1 ]=> '(+ 2 3 4)
;Value: (+ 2 3 4)
1 ]=> (+ 2 3 4)
;Value: 9
When you quote a code, the result will be data that represent that code. So, when you want to work with data that represents a program you quote that program. This is also valid for atomic expressions, not only for lists:
1 ]=> 'code
;Value: code
1 ]=> '10
;Value: 10
1 ]=> '"ok"
;Value: "ok"
1 ]=> code
;Unbound variable: code
Supposing you want to create a programming language embedded in lisp -- you will work with programs that are quoted in scheme (like '(+ 2 3)) and that are interpreted as code in the language you create, by giving programs a semantic interpretation. In this case you need to use quote to keep the data, otherwise it will be evaluated in external language.
When we want to pass an argument itself instead of passing the value of the argument then we use quote. It is mostly related to the procedure passing during using lists, pairs and atoms
which are not available in C programming Language ( most people start programming using C programming, Hence we get confused)
This is code in Scheme programming language which is a dialect of lisp and I guess you can understand this code.
(define atom? ; defining a procedure atom?
(lambda (x) ; which as one argument x
(and (not (null? x)) (not(pair? x) )))) ; checks if the argument is atom or not
(atom? '(a b c)) ; since it is a list it is false #f
The last line (atom? 'abc) is passing abc as it is to the procedure to check if abc is an atom or not, but when you pass(atom? abc) then it checks for the value of abc and passses the value to it. Since, we haven't provided any value to it
Quote returns the internal representation of its arguments. After plowing through way too many explanations of what quote doesn't do, that's when the light-bulb went on. If the REPL didn't convert function names to UPPER-CASE when I quoted them, it might not have dawned on me.
So. Ordinary Lisp functions convert their arguments into an internal representation, evaluate the arguments, and apply the function. Quote converts its arguments to an internal representation, and just returns that. Technically it's correct to say that quote says, "don't evaluate", but when I was trying to understand what it did, telling me what it doesn't do was frustrating. My toaster doesn't evaluate Lisp functions either; but that's not how you explain what a toaster does.
Anoter short answer:
quote means without evaluating it, and backquote is quote but leave back doors.
A good referrence:
Emacs Lisp Reference Manual make it very clear
9.3 Quoting
The special form quote returns its single argument, as written, without evaluating it. This provides a way to include constant symbols and lists, which are not self-evaluating objects, in a program. (It is not necessary to quote self-evaluating objects such as numbers, strings, and vectors.)
Special Form: quote object
This special form returns object, without evaluating it.
Because quote is used so often in programs, Lisp provides a convenient read syntax for it. An apostrophe character (‘'’) followed by a Lisp object (in read syntax) expands to a list whose first element is quote, and whose second element is the object. Thus, the read syntax 'x is an abbreviation for (quote x).
Here are some examples of expressions that use quote:
(quote (+ 1 2))
⇒ (+ 1 2)
(quote foo)
⇒ foo
'foo
⇒ foo
''foo
⇒ (quote foo)
'(quote foo)
⇒ (quote foo)
9.4 Backquote
Backquote constructs allow you to quote a list, but selectively evaluate elements of that list. In the simplest case, it is identical to the special form quote (described in the previous section; see Quoting). For example, these two forms yield identical results:
`(a list of (+ 2 3) elements)
⇒ (a list of (+ 2 3) elements)
'(a list of (+ 2 3) elements)
⇒ (a list of (+ 2 3) elements)
The special marker ‘,’ inside of the argument to backquote indicates a value that isn’t constant. The Emacs Lisp evaluator evaluates the argument of ‘,’, and puts the value in the list structure:
`(a list of ,(+ 2 3) elements)
⇒ (a list of 5 elements)
Substitution with ‘,’ is allowed at deeper levels of the list structure also. For example:
`(1 2 (3 ,(+ 4 5)))
⇒ (1 2 (3 9))
You can also splice an evaluated value into the resulting list, using the special marker ‘,#’. The elements of the spliced list become elements at the same level as the other elements of the resulting list. The equivalent code without using ‘`’ is often unreadable. Here are some examples:
(setq some-list '(2 3))
⇒ (2 3)
(cons 1 (append some-list '(4) some-list))
⇒ (1 2 3 4 2 3)
`(1 ,#some-list 4 ,#some-list)
⇒ (1 2 3 4 2 3)

What would the consequences be of allowing special forms to be treated as normal values?

I discovered that special forms can not be passed as arguments or saved in variables, both in Clojure:
user=> (defn my-func
[op]
(op 1 2 3))
#'user/my-func
user=> (my-func +)
6
user=> (my-func if)
java.lang.Exception: Unable to resolve symbol: if in this context (NO_SOURCE_FILE:5)
user=> (def my-if if)
java.lang.Exception: Unable to resolve symbol: if in this context (NO_SOURCE_FILE:9)
and in Racket/Scheme:
> (define (my-func op)
(op 1 2 3))
> (my-func +)
6
> (my-func if)
if: bad syntax in: if
> (define my-if if)
*if: bad syntax in: if
That's cool, I'm fine with that, I know that I can just write a function to wrap a special form and then pass that function.
But I'd like to know why Lisps have this restriction, and what negative consequences allowing this would have. Are they any major dialects that do allow that?
It makes evaluation more complicated and compilation hard.
If you have a form (a b c), then you need to resolve at runtime the value of a and then apply it somehow to the forms b and c.
The simpler model of a few limited number of special forms and otherwise strict evaluation is then gone.
See also: FEXPR
Special forms are not functions: functions take values as arguments, whereas special forms take forms. For example, look at your example of if:
(if 1 2 3)
Okay, that's easy, since 2 and 3 are already values. But what about this?
(define (modadd a b n)
(if (zero? n) #f (modulo (+ a b) n)))
In this case, the if is actually receiving the #f and (modulo (+ a b) n) as forms, not as values. And this is important! (modulo x n) will fail if n is 0: that is why it's left unevaluated until we know that n is not 0.
The trouble with being able to pass special forms as first-class objects is that higher-order functions cannot call those objects using predictable semantics: is it a function object, so that you're passing in values, or is it a special form, so that you're passing in forms? This gets to be a huge mess.
Yes, you can write a wrapper function, like my modadd, that encapsulates if. However, there is no way you can just reimplement if as a function, while still preserving the only-evaluate-one-of-the-branches behaviour.

How to avoid anaphoric macro in Clojure?

Take this (simplified) example:
(defmacro make [v & body]
`(let [~'nv ~(some-calc v)]
~(map #(if (= % :value) 'nv %) body)))
Right now the symbol nv is hardcoded. Is there a way to somehow gensym nv and still being able to use it in the map function?
By the way, is this actually an anaphoric macro?
The answer is contained within the question: just use gensym like you would if Clojure didn't have auto-gensyms.
(defmacro make [v & body]
(let [value-sym (gensym)]
`(let [~value-sym ~(some-calc v)]
~#(replace {:value value-sym} body))))
Note that I'm not sure whether you really want ~ or ~# here - it depends if body is supposed to be a sequence of expressions to execute in the let, or a sequence of arguments to a single function call. But ~# would be a lot more intuitive/normal, so that's what I'm going to guess.
Whether this macro is anaphoric is a little questionable: definitely introducing nv into the calling scope was, but it was basically unintentional so I would say no. In my revised version, we're no longer introducing nv or anything like it, but we are "magically" replacing :value with v. We only do that at the topmost level of the body, though, so it's not like introducing a real scope - I'd say it's more like making the client's code unexpectedly break in corner cases.
For an example of how this deceptive behavior can crop up, imagine that one of the elements of body is (inc :value). It won't get replaced by the macro, and will expand to (inc :value), which never succeeds. So instead I'd recommend a real anaphoric macro, which introduces a real scope for a symbol. Something like
(defmacro make [v & body]
`(let [~'the-value ~(some-calc v)]
~#body))
And then the caller can just use the-value in their code, and it behaves just like a real, regular local binding: your macro introduces it by magic, but it doesn't have any other special tricks.
It's not actually an anaphoric macro as I understand it.
An anaphoric equivalent would have give you a syntax like:
(make foo 1 2 3 4 it 6 7 it 8 9)
i.e. the symbol it has been defined so that it can be used inside the body of the make macro.
I'm not sure precisely if this is what you want because I don't have enough context on how this macro is going to be used, but you could implement the above like:
(defmacro make [v & body]
`(let [~'it ~v]
(list ~#body)))
(make (* 10 10) 1 2 3 4 it 6 7 it 8 9)
=> (1 2 3 4 100 6 7 100 8 9)
Alternatively, if you're not really trying to create new syntax and just want to replace :value in some collection, then you don't really need a macro: it would be better to just use replace:
(replace {:value (* 10 10)} [1 2 :value 3 4])
=> [1 2 100 3 4]
In your example, both some-calc and map happen during macroexpansion, not at runtime, thus nv does not need to be let anyway. The macro itself is not written properly, irrespective of anything to do with symbol capture.
One approach is to use a dynamic binding.
(declare ^:dynamic *nv*)
(defmacro make [v & body]
`(binding [*nv* ~(some-calc v)]
~(map #(if (= % :value) *nv* %) body)))
In practice, dynamic variables suck when their scope is too wide (it's harder to test and debug programs etc.), but in cases like this where the scope is confined to the local calling context where an anaphor is needed, they can be quite handy.
An interesting aspect of this use is that it's kind of the inverse of a common idiom of using a macro to hide a dynamic binding (many in the with-* style). In this idiom (which as far as I know is not all that common) the binding is used to expose something hidden by the macro.

Why should I use 'apply' in Clojure?

This is what Rich Hickey said in one of the blog posts but I don't understand the motivation in using apply. Please help.
A big difference between Clojure and CL is that Clojure is a Lisp-1, so funcall is not needed, and apply is only used to apply a function to a runtime-defined collection of arguments. So, (apply f [i]) can be written (f i).
Also, what does he mean by "Clojure is Lisp-1" and funcall is not needed? I have never programmed in CL.
Thanks
You would use apply, if the number of arguments to pass to the function is not known at compile-time (sorry, don't know Clojure syntax all that well, resorting to Scheme):
(define (call-other-1 func arg) (func arg))
(define (call-other-2 func arg1 arg2) (func arg1 arg2))
As long as the number of arguments is known at compile time, you can pass them directly as is done in the example above. But if the number of arguments is not known at compile-time, you cannot do this (well, you could try something like):
(define (call-other-n func . args)
(case (length args)
((0) (other))
((1) (other (car args)))
((2) (other (car args) (cadr args)))
...))
but that becomes a nightmare soon enough. That's where apply enters the picture:
(define (call-other-n func . args)
(apply other args))
It takes whatever number of arguments are contained in the list given as last argument to it, and calls the function passed as first argument to apply with those values.
The terms Lisp-1 and Lisp-2 refer to whether functions are in the same namespace as variables.
In a Lisp-2 (that is, 2 namespaces), the first item in a form will be evaluated as a function name — even if it's actually the name of a variable with a function value. So if you want to call a variable function, you have to pass the variable to another function.
In a Lisp-1, like Scheme and Clojure, variables that evaluate to functions can go in the initial position, so you don't need to use apply in order to evaluate it as a function.
apply basically unwraps a sequence and applies the function to them as individual arguments.
Here is an example:
(apply + [1 2 3 4 5])
That returns 15. It basically expands to (+ 1 2 3 4 5), instead of (+ [1 2 3 4 5]).
You use apply to convert a function that works on several arguments to one that works on a single sequence of arguments. You can also insert arguments before the sequence. For example, map can work on several sequences. This example (from ClojureDocs) uses map to transpose a matrix.
user=> (apply map vector [[:a :b] [:c :d]])
([:a :c] [:b :d])
The one inserted argument here is vector. So the apply expands to
user=> (map vector [:a :b] [:c :d])
Cute!
PS To return a vector of vectors instead of a sequence of vectors, wrap the whole thing in vec:
user=> (vec (apply map vector [[:a :b] [:c :d]]))
While we're here, vec could be defined as (partial apply vector), though it isn't.
Concerning Lisp-1 and Lisp-2: the 1 and 2 indicate the number of things a name can denote in a given context. In a Lisp-2, you can have two different things (a function and a variable) with the same name. So, wherever either might be valid, you need to decorate your program with something to indicate which you mean. Thankfully, Clojure (or Scheme ...) allows a name to denote just one thing, so no such decorations are necessary.
The usual pattern for apply type operations is to combine a function provided at runtime with a set of arguments, ditto.
I've not done enough with clojure to be able to be confident about the subtleties for that particular language to tell whether the use of apply in that case would be strictly necessary.
Apply is useful with protocols, especially in conjunction with threading macros. I just discovered this. Since you can't use the & macro to expand interface arguments at compile time, you can apply an unpredictably sized vector instead.
So I use this, for instance, as part of an interface between a record holding some metadata about a particular xml file and the file itself.
(query-tree [this forms]
(apply xml-> (text-id-to-tree this) forms)))
text-id-to-tree is another method of this particular record that parses a file into an xml zipper. In another file, I extend the protocol with a particular query that implements query-tree, specifying a chain of commands to be threaded through the xml-> macro:
(tags-with-attrs [this]
(query-tree this [zf/descendants zip/node (fn [node] [(map #(% node) [:tag :attrs])])])
(note: this query by itself will return a lot of "nil" results for tags that don't have
attributes. Filter and reduce for a clean list of unique values).
zf, by the way, refers to clojure.contrib.zip-filter, and zip to clojure.zip. The xml-> macro is from the clojure.contrib.zip-filter.xml library, which I :use

When to use ' (or quote) in Lisp?

After making it through the major parts of an introductory Lisp book, I still couldn't understand what the special operator (quote) (or equivalent ') function does, yet this has been all over Lisp code that I've seen.
What does it do?
Short answer
Bypass the default evaluation rules and do not evaluate the expression (symbol or s-exp), passing it along to the function exactly as typed.
Long Answer: The Default Evaluation Rule
When a regular (I'll come to that later) function is invoked, all arguments passed to it are evaluated. This means you can write this:
(* (+ a 2)
3)
Which in turn evaluates (+ a 2), by evaluating a and 2. The value of the symbol a is looked up in the current variable binding set, and then replaced. Say a is currently bound to the value 3:
(let ((a 3))
(* (+ a 2)
3))
We'd get (+ 3 2), + is then invoked on 3 and 2 yielding 5. Our original form is now (* 5 3) yielding 15.
Explain quote Already!
Alright. As seen above, all arguments to a function are evaluated, so if you would like to pass the symbol a and not its value, you don't want to evaluate it. Lisp symbols can double both as their values, and markers where you in other languages would have used strings, such as keys to hash tables.
This is where quote comes in. Say you want to plot resource allocations from a Python application, but rather do the plotting in Lisp. Have your Python app do something like this:
print("'(")
while allocating:
if random.random() > 0.5:
print(f"(allocate {random.randint(0, 20)})")
else:
print(f"(free {random.randint(0, 20)})")
...
print(")")
Giving you output looking like this (slightly prettyfied):
'((allocate 3)
(allocate 7)
(free 14)
(allocate 19)
...)
Remember what I said about quote ("tick") causing the default rule not to apply? Good. What would otherwise happen is that the values of allocate and free are looked up, and we don't want that. In our Lisp, we wish to do:
(dolist (entry allocation-log)
(case (first entry)
(allocate (plot-allocation (second entry)))
(free (plot-free (second entry)))))
For the data given above, the following sequence of function calls would have been made:
(plot-allocation 3)
(plot-allocation 7)
(plot-free 14)
(plot-allocation 19)
But What About list?
Well, sometimes you do want to evaluate the arguments. Say you have a nifty function manipulating a number and a string and returning a list of the resulting ... things. Let's make a false start:
(defun mess-with (number string)
'(value-of-number (1+ number) something-with-string (length string)))
Lisp> (mess-with 20 "foo")
(VALUE-OF-NUMBER (1+ NUMBER) SOMETHING-WITH-STRING (LENGTH STRING))
Hey! That's not what we wanted. We want to selectively evaluate some arguments, and leave the others as symbols. Try #2!
(defun mess-with (number string)
(list 'value-of-number (1+ number) 'something-with-string (length string)))
Lisp> (mess-with 20 "foo")
(VALUE-OF-NUMBER 21 SOMETHING-WITH-STRING 3)
Not Just quote, But backquote
Much better! Incidently, this pattern is so common in (mostly) macros, that there is special syntax for doing just that. The backquote:
(defun mess-with (number string)
`(value-of-number ,(1+ number) something-with-string ,(length string)))
It's like using quote, but with the option to explicitly evaluate some arguments by prefixing them with comma. The result is equivalent to using list, but if you're generating code from a macro you often only want to evaluate small parts of the code returned, so the backquote is more suited. For shorter lists, list can be more readable.
Hey, You Forgot About quote!
So, where does this leave us? Oh right, what does quote actually do? It simply returns its argument(s) unevaluated! Remember what I said in the beginning about regular functions? Turns out that some operators/functions need to not evaluate their arguments. Such as IF -- you wouldn't want the else branch to be evaluated if it wasn't taken, right? So-called special operators, together with macros, work like that. Special operators are also the "axiom" of the language -- minimal set of rules -- upon which you can implement the rest of Lisp by combining them together in different ways.
Back to quote, though:
Lisp> (quote spiffy-symbol)
SPIFFY-SYMBOL
Lisp> 'spiffy-symbol ; ' is just a shorthand ("reader macro"), as shown above
SPIFFY-SYMBOL
Compare to (on Steel-Bank Common Lisp):
Lisp> spiffy-symbol
debugger invoked on a UNBOUND-VARIABLE in thread #<THREAD "initial thread" RUNNING {A69F6A9}>:
The variable SPIFFY-SYMBOL is unbound.
Type HELP for debugger help, or (SB-EXT:QUIT) to exit from SBCL.
restarts (invokable by number or by possibly-abbreviated name):
0: [ABORT] Exit debugger, returning to top level.
(SB-INT:SIMPLE-EVAL-IN-LEXENV SPIFFY-SYMBOL #<NULL-LEXENV>)
0]
Because there is no spiffy-symbol in the current scope!
Summing Up
quote, backquote (with comma), and list are some of the tools you use to create lists, that are not only lists of values, but as you seen can be used as lightweight (no need to define a struct) data structures!
If you wish to learn more, I recommend Peter Seibel's book Practical Common Lisp for a practical approach to learning Lisp, if you're already into programming at large. Eventually on your Lisp journey, you'll start using packages too. Ron Garret's The Idiot's Guide to Common Lisp Packages will give you good explanation of those.
Happy hacking!
It says "don't evaluate me". For example, if you wanted to use a list as data, and not as code, you'd put a quote in front of it. For example,
(print '(+ 3 4)) prints "(+ 3 4)", whereas
(print (+ 3 4)) prints "7"
Other people have answered this question admirably, and Matthias Benkard brings up an excellent warning.
DO NOT USE QUOTE TO CREATE LISTS THAT YOU WILL LATER MODIFY. The spec allows the compiler to treat quoted lists as constants. Often, a compiler will optimize constants by creating a single value for them in memory and then referencing that single value from all locations where the constant appears. In other words, it may treat the constant like an anonymous global variable.
This can cause obvious problems. If you modify a constant, it may very well modify other uses of the same constant in completely unrelated code. For example, you may compare some variable to '(1 1) in some function, and in a completely different function, start a list with '(1 1) and then add more stuff to it. Upon running these functions, you may find that the first function doesn't match things properly anymore, because it's now trying to compare the variable to '(1 1 2 3 5 8 13), which is what the second function returned. These two functions are completely unrelated, but they have an effect on each other because of the use of constants. Even crazier bad effects can happen, like a perfectly normal list iteration suddenly infinite looping.
Use quote when you need a constant list, such as for comparison. Use list when you will be modifying the result.
One answer to this question says that QUOTE “creates list data structures”. This isn't quite right. QUOTE is more fundamental than this. In fact, QUOTE is a trivial operator: Its purpose is to prevent anything from happening at all. In particular, it doesn't create anything.
What (QUOTE X) says is basically “don't do anything, just give me X.” X needn't be a list as in (QUOTE (A B C)) or a symbol as in (QUOTE FOO). It can be any object whatever. Indeed, the result of evaluating the list that is produced by (LIST 'QUOTE SOME-OBJECT) will always just return SOME-OBJECT, whatever it is.
Now, the reason that (QUOTE (A B C)) seems as if it created a list whose elements are A, B, and C is that such a list really is what it returns; but at the time the QUOTE form is evaluated, the list has generally already been in existence for a while (as a component of the QUOTE form!), created either by the loader or the reader prior to execution of the code.
One implication of this that tends to trip up newbies fairly often is that it's very unwise to modify a list returned by a QUOTE form. Data returned by QUOTE is, for all intents and purposes, to be considered as part of the code being executed and should therefore be treated as read-only!
The quote prevents execution or evaluation of a form, turning it instead into data. In general you can execute the data by then eval'ing it.
quote creates list data structures, for example, the following are equivalent:
(quote a)
'a
It can also be used to create lists (or trees):
(quote (1 2 3))
'(1 2 3)
You're probably best off getting an introductary book on lisp, such as Practical Common Lisp (which is available to read on-line).
In Emacs Lisp:
What can be quoted ?
Lists and symbols.
Quoting a number evaluates to the number itself:
'5 is the same as 5.
What happens when you quote lists ?
For example:
'(one two) evaluates to
(list 'one 'two) which evaluates to
(list (intern "one") (intern ("two"))).
(intern "one") creates a symbol named "one" and stores it in a "central" hash-map, so anytime you say 'one then the symbol named "one" will be looked up in that central hash-map.
But what is a symbol ?
For example, in OO-languages (Java/Javascript/Python) a symbol could be represented as an object that has a name field, which is the symbol's name like "one" above, and data and/or code can be associated with it this object.
So an symbol in Python could be implemented as:
class Symbol:
def __init__(self,name,code,value):
self.name=name
self.code=code
self.value=value
In Emacs Lisp for example a symbol can have 1) data associated with it AND (at the same time - for the same symbol) 2) code associated with it - depending on the context, either the data or the code gets called.
For example, in Elisp:
(progn
(fset 'add '+ )
(set 'add 2)
(add add add)
)
evaluates to 4.
Because (add add add) evaluates as:
(add add add)
(+ add add)
(+ 2 add)
(+ 2 2)
4
So, for example, using the Symbol class we defined in Python above, this add ELisp-Symbol could be written in Python as Symbol("add",(lambda x,y: x+y),2).
Many thanks for folks on IRC #emacs for explaining symbols and quotes to me.
Code is data and data is code. There is no clear distinction between them.
This is a classical statement any lisp programmer knows.
When you quote a code, that code will be data.
1 ]=> '(+ 2 3 4)
;Value: (+ 2 3 4)
1 ]=> (+ 2 3 4)
;Value: 9
When you quote a code, the result will be data that represent that code. So, when you want to work with data that represents a program you quote that program. This is also valid for atomic expressions, not only for lists:
1 ]=> 'code
;Value: code
1 ]=> '10
;Value: 10
1 ]=> '"ok"
;Value: "ok"
1 ]=> code
;Unbound variable: code
Supposing you want to create a programming language embedded in lisp -- you will work with programs that are quoted in scheme (like '(+ 2 3)) and that are interpreted as code in the language you create, by giving programs a semantic interpretation. In this case you need to use quote to keep the data, otherwise it will be evaluated in external language.
When we want to pass an argument itself instead of passing the value of the argument then we use quote. It is mostly related to the procedure passing during using lists, pairs and atoms
which are not available in C programming Language ( most people start programming using C programming, Hence we get confused)
This is code in Scheme programming language which is a dialect of lisp and I guess you can understand this code.
(define atom? ; defining a procedure atom?
(lambda (x) ; which as one argument x
(and (not (null? x)) (not(pair? x) )))) ; checks if the argument is atom or not
(atom? '(a b c)) ; since it is a list it is false #f
The last line (atom? 'abc) is passing abc as it is to the procedure to check if abc is an atom or not, but when you pass(atom? abc) then it checks for the value of abc and passses the value to it. Since, we haven't provided any value to it
Quote returns the internal representation of its arguments. After plowing through way too many explanations of what quote doesn't do, that's when the light-bulb went on. If the REPL didn't convert function names to UPPER-CASE when I quoted them, it might not have dawned on me.
So. Ordinary Lisp functions convert their arguments into an internal representation, evaluate the arguments, and apply the function. Quote converts its arguments to an internal representation, and just returns that. Technically it's correct to say that quote says, "don't evaluate", but when I was trying to understand what it did, telling me what it doesn't do was frustrating. My toaster doesn't evaluate Lisp functions either; but that's not how you explain what a toaster does.
Anoter short answer:
quote means without evaluating it, and backquote is quote but leave back doors.
A good referrence:
Emacs Lisp Reference Manual make it very clear
9.3 Quoting
The special form quote returns its single argument, as written, without evaluating it. This provides a way to include constant symbols and lists, which are not self-evaluating objects, in a program. (It is not necessary to quote self-evaluating objects such as numbers, strings, and vectors.)
Special Form: quote object
This special form returns object, without evaluating it.
Because quote is used so often in programs, Lisp provides a convenient read syntax for it. An apostrophe character (‘'’) followed by a Lisp object (in read syntax) expands to a list whose first element is quote, and whose second element is the object. Thus, the read syntax 'x is an abbreviation for (quote x).
Here are some examples of expressions that use quote:
(quote (+ 1 2))
⇒ (+ 1 2)
(quote foo)
⇒ foo
'foo
⇒ foo
''foo
⇒ (quote foo)
'(quote foo)
⇒ (quote foo)
9.4 Backquote
Backquote constructs allow you to quote a list, but selectively evaluate elements of that list. In the simplest case, it is identical to the special form quote (described in the previous section; see Quoting). For example, these two forms yield identical results:
`(a list of (+ 2 3) elements)
⇒ (a list of (+ 2 3) elements)
'(a list of (+ 2 3) elements)
⇒ (a list of (+ 2 3) elements)
The special marker ‘,’ inside of the argument to backquote indicates a value that isn’t constant. The Emacs Lisp evaluator evaluates the argument of ‘,’, and puts the value in the list structure:
`(a list of ,(+ 2 3) elements)
⇒ (a list of 5 elements)
Substitution with ‘,’ is allowed at deeper levels of the list structure also. For example:
`(1 2 (3 ,(+ 4 5)))
⇒ (1 2 (3 9))
You can also splice an evaluated value into the resulting list, using the special marker ‘,#’. The elements of the spliced list become elements at the same level as the other elements of the resulting list. The equivalent code without using ‘`’ is often unreadable. Here are some examples:
(setq some-list '(2 3))
⇒ (2 3)
(cons 1 (append some-list '(4) some-list))
⇒ (1 2 3 4 2 3)
`(1 ,#some-list 4 ,#some-list)
⇒ (1 2 3 4 2 3)