The clojure macro is a difficult point for me,
here is an macro example took from "Pratical Clojure":
(defmacro triple-do [form]
(list 'do form form form))
user=> (triple-do (println "test"))
test
test
test
nil
This triple-do works well
And I think the following version should work but not
(defmacro triple-do [form]
(do form form form))
user=> (triple-do (println "test"))
test
nil
why does it just print once?
and the following confuses me extremely
(defmacro test-macro [form] (do form (println "hard code test")))
user=> (test-macro (println "hello"))
hard code test
nil
why "hello" not show in console?
A macro is a function which returns a Clojure s-expression which is then compiled, or expanded again if the macro returns a macro. This process of replacement repeats recursivly until no macro's remain, then the final code is evaluated. It helps to think carefully about what runs while the macros are expanding compared to what the finally produced code will run.
The macroexpand-1 function can be really helpful by showing you what a macro expands:
user> (macroexpand-1 '(test-macro (println "hello")))
hard code test
nil
from this you can see that the print statement is happening while the macro is expanding. If you add a syntax quote before your do, the macro may make more sense.
user> (defmacro test-macro [form] `(do ~form (println "hard code test")))
#'user/test-macro
user> (macroexpand-1 '(test-macro (println "hello")))
(do (println "hello") (clojure.core/println "hard code test"))
In this case the print runs after the macro finishes expanding.
It will help to look at your examples through the lens of macroexpand, an essential tool for debugging macros.
Example 1:
(defmacro triple-do [form]
(list 'do form form form))
user=> (triple-do (println "test"))
Remember macros are supposed to return a list that will be executed at runtime. Let's see what this macro returns:
(macroexpand '(triple-do (println "test")))
;; (do (println "test") (println "test") (println "test"))
So it doesn't execute the code but rather returns a list that represents the code that will be executed once the macro is expanded. This is similar to trying the following snippet at the REPL:
(+ 1 2 3)
;; 6
(list '+ 1 2 3)
;; (+ 1 2 3)
With this in mind, let's turn to Example 2:
(defmacro triple-do [form]
(do form form form))
user=> (triple-do (println "test"))
Notice how the macro now doesn't return a list. It simply executes the do form which returns the last statement which is the form passed in. This can be easily seen by expanding the macro:
(macroexpand '(triple-do (println "test")))
;; (println "test")
That's why you end up with a single print statement.
This should give you a clue about Example 3:
(defmacro test-macro [form] (do form (println "hard code test")))
user=> (test-macro (println "hello"))
It's a bit trickier but let's expand it nevertheless:
(macroexpand '(test-macro (println "hello")))
;; hard code test <= this gets print before the macro fully expands
;; nil <= the expansion yields nil
Again, since you're not returning a list but rather simply executing a do form, it simply runs the println call from within the macro and since println returns nil, that becomes the result of the expansion.
To illustrate my point, this is how you'd have to rewrite your macro in order to achieve the desired behaviour:
(defmacro test-macro [form] (list 'do form (println "hard code test")))
(test-macro (println "hello"))
;; hard code test
;; hello
I hope this clears things for you.
Just remember this: macros are supposed to return lists that represent the code you'd like to be executed at runtime.
A macro must return, at compile time, the list that will take its place in the code.
Since do takes any number of arguments and returns the last, in each of your cases the macro expands to the last form in the do block.
The original returned a list with do as the first element, so rather than returning the last element in the block, it expanded to the whole block.
Related
I would like a more detailed explanation of how macro expansion works, at least in Emacs Lisp but an overview of other Lisps would be appreciated. The way I usually see it explained is that the arguments of the macro are passed unevaluated to the body, which is then executed and returns a new LISP form. However, if I do
(defun check-one (arg)
(eq arg 1))
(defmacro check-foo (checker foo)
(if (checker 1)
`(,foo "yes")
`(,foo "no")))
I would expect
(check-foo check-one print)
to first expand to
(if (check-one 1)
`(print "yes")
`(print "no))
and then finally to
(print "yes")
but instead I get a "checker" function is void error. On the other hand, if I had defined
(defmacro check-foo (checker foo)
(if (funcall checker 1)
`(,foo "yes")
`(,foo "no")))
then I would have the expected behavior. So the expressions do get replaced in the body unevaluated, but for some reason functions do not work? What is the step-by-step procedure the interpreter follows when macroexpanding? Is there a good text-book that explains this rigorously?
Macros are functions ...
A good way to think about macros is that they are simply functions, like any other function.
... which operate on source code
But they are functions whose arguments are source code, and whose value is also source code.
Looking at macro functions
Macros being functions is not quite explicit in elisp: some of the lower-level functionality is, I think, not exposed. But in Common Lisp this is quite literally how macros are implemented: a macro has an associated function, and this function gets called to expand the macro, with its value being the new source code. For instance, if you are so minded you could write macros in Common Lisp like this.
(defun expand-fn (form environment)
;; not talking about environment
(declare (ignore environment))
(let ((name (second form))
(arglist (third form))
(body (cdddr form)))
`(function (lambda ,arglist
(block ,name
,#body)))))
(setf (macro-function 'fn) #'expand-fn)
And now fn is a macro which will construct a function which 'knows its name', so you could write
(fn foo (x) ... (return-from foo x) ...)
which turns into
(function (lambda (x) (block foo ... (return-from foo x))))
In Common Lisp, defmacro is then itself a macro which arranges for a suitable macro function to be installed and also deals with making the macro available at compile time &c.
In elisp, it looks as if this lower layer is not specified by the language, but I think it's safe to assume that things work the same way.
So then the job of a macro is to take a bunch of source code and compute from it another bunch of source code which is the expansion of the macro. And of course the really neat trick is that, because source code (both arguments and values) is represented as s-expressions, Lisp is a superb language for manipulating s-expressions, you can write macros in Lisp itself.
Macroexpansion
There are a fair number of fiddly corner cases here such as local macros and so on. But here is, pretty much, how this works.
Start with some form <f>:
If <f> is (<a> ...) where <a> is a symbol, check for a macro function for <a>. If it has one, call it on the whole form, and call the value it returns <f'>: now simply recurse on <f'>.
If <f> is (<a> ...) where <a> is a symbol which names a special operator (something like if) then recurse on the subforms of the special operator which its rules say must be macroexpanded. As an example, in a form like (if <x> <y> <z>) all of <x>, <y>, & <z> need to be macroexpanded, while in (setq <a> <b>), only <b> would be subject to macroexpansion, and so on: these rules are hard-wired, which is why special operators are special.
If <f> is (<a> ...) where <a> is a symbol which is neither of the above cases, then it's a function call, and the forms in the body of the form are macroexpanded, and that's it.
If <f> is ((lambda (...) ...) ...) then the forms in the body of the lambda (but not its arguments!) are macroexpanded and then the case is the same as the last one.
Finally <f> might not be a compound form: nothing to do here.
I think that's all the cases. This is not a complete description of the process because there are complications like local macros and so on. But it's enough I think.
Order of macroexpansion
Note that macroexpansion happens 'outside in': a form like (a ...) is expanded until you get something which isn't a macro form, and only then is the body, perhaps, expanded. That's because, until the macro is completely expanded, you have no idea which, if any, of the subforms are even eligible for macroexpansion.
Your code
My guess is that what you want to happen is that (check-foo bog foo) should turn into (if (bog 1) (foo yes) (foo no)). So the way to get this is that this form is what the macro function needs to return. We could write this using the CL low-level facilities:
(defun check-foo-expander (form environment)
;; form is like (check-foo pred-name function-name)
(declare (ignore environment)) ;still not talking about environment
`(if (,(second form) 1)
(,(third form) "yes")
(,(third form) "no")))
And we can check:
> (check-foo-expander '(check-foo bog foo) nil)
(if (bog 1) (foo "yes") (foo "no"))
And then install it as a macro:
> (setf (macro-function 'check-foo) #'check-foo-expander)
And now
> (check-foo evenp print)
"no"
"no"
> (check-foo oddp print)
"yes"
"yes"
But it's easier to write it using defmacro:
(defmacro check-foo (predicate function)
`(if (,predicate 1)
(,function "yes")
(,function "no")))
This is the same thing (more-or-less), but easier to read.
I currently have a macro defined as follows:
(defmacro some-macro (generic-name (&rest args) &body body)
...)
I would now like to add an extra parameter to this macro, a kind of flag which a user can optionally provide and which would alter the behavior of the macro. The below code should be possible:
(some-macro some-name (arg1 arg2) (print (+ arg1 arg2)))
(some-macro :flag some-name (arg1 arg2) (print (special-+ arg1 arg2)))
I intentionally just made a conceptual example to focus on what's important instead of the real macro I'm trying to implement. The real a macro is different and more complex. As you can see, the macro should now be callable with both the :flag argument (flag can be any word, as long as its prefix is ':') and without the flag argument. Is there a way to do this without positioning the &optional keyword at the end of the parameter list (ie. it really needs to be at that first position).
&optional can only be at the end of the positional arguments (you can have &rest or &body after it). Even if you could put it earlier, if there's also &rest or &body, how would it know whether you provided the optional argument? E.g. if the lambda list were
(&optional arg1 arg2 &rest rest-args)
and the call were
(func-name 1 2 3 4 5)
it could either be arg1 = 1, arg2 = 2, rest-args = (3 4 5) or arg1 = NIL, arg2 = 1, rest-args = (2 3 4 5).
You'll need to define the macro to take a single &rest argument. Then you can check whether the first argument is a keyword or not, update the argument list to add the default, and then parse it with destructuring-bind.
(defmacro some-macro (&rest all-args)
(unless (keywordp (first all-args))
(push nil all-args)) ;; flag defaults to NIL
(destructuring-bind (flag generic-name (&rest args) &body body) all-args
...))
(some-macro :flag some-name (arg1 arg2) (print (special-+ arg1 arg2)))
(some-macro some-name (arg1 arg2) (print (+ arg1 arg2)))
One can only guess what's useful, since the design of the macro depends a bit on more context: what is it actually used for.
For example in CLOS we write
(defmethod foo :around ((a class-a)) ...)
The name comes first, then zero or more method-qualifiers (here :around and then the arglist. Putting a flag in front of a name would be strange in typical defining macros (those who begin with def).
For that we would need to write the macro-destructuring of the arglist ourselves, since it does not match a built-in macro arglist pattern.
(defmacro some-macro (name &rest args)
(let* ((qualifiers (loop for arg in args
until (listp arg)
collect (pop args)))
(arg-list (pop args))
(body args))
... ; return something for the example
))
In other macros we might write
(some-macro some-name (arg1 arg2 :flag)
(print (special-+ arg1 arg2)))
(defmacro some-macro (some-name (arg1 arg2 &optional flag) &body body)
...)
Similar for example to
(with-input-from-string (stream string :start 10)
(... ))
Though above uses keywords, not optionals.
Or we might want to write:
(some-macro some-name (arg1 arg2) (:flag1)
(print (special-+ arg1 arg2)))
(defmacro some-macro (some-name (&rest args) (&rest flags) &body body)
...)
If there are only three flags one can also generate three different macros with different names and remove the flag.
One thing to think about when designing things like macros (and, remember, when you are designing macros, you are designing a programming language) is how people expect to read the language. If the programming language you are designing is a mild superset of CL, or is otherwise very close to CL, then you may want to not violate the expectations CL programmers have when reading CL code, or more generally that Lisp programmers have when reading Lisp code.
[Note that most of what follows is opinion: people obviously have different opinions – these are mine, they're not more right than anyone else's.]
So, what are those expectations? Well, they may include the following two:
people read CL left-to-right, so the things at the left-hand end of a form tend to be more visually important;
many existing forms in CL look like (<operator> <thing> ...) – the first two subforms in the form are by far the most interesting, and the second is often more interesting than the first. Think about (defun foo ...), (dolist (x ...) ...), (let ((x y) ...) ...).
An example of something which violates these expectations is an object system which works explicitly with message-passing, using some send operation (I think Old Flavors did this, and I think that New Flavors didn't, but my memory is vague now). Code written using these looks like (send <object> <message> ...): the first word in many forms is send. That means that this visually-important place for reading code has been entirely wasted as it is always the same word and the important places are now the second and third subform. Well, instead, we could just omit the whole send thing and write (message object ...) or (object message ...). CLOS takes essentially the first of these options, where a 'message' is a generic function and of course generic functions can specialise on more than one argument which breaks the whole message-passing paradigm. But you can write CLOS as if it was message-passing, and it works, and it means that it agrees with how a lot of other CL code looks.
So, OK let's look at two cases of your macro:
(some-macro some-name ...)
This is fine.
(some-macro :flag some-name ...)
But this has filled the visual second-position slot with something which is not whatever the macro is about: it's just some optional argument. The interesting thing is now the third position.
Well, how could we fix this? It turns out there is a good example which already exists in CL: defstruct. defstruct has two basic cases:
(defstruct structure-name
...)
and
(defstruct (structure-name ...)
...)
Both of these meet the first-two-positions-matter-most requirement, while allowing optional arguments and clearly visually indicating when they are used.
(Aside: defclasss does things differently by putting the options at the end as in:
(defclass name (...supers...)
(...slot specifications...)
...options...))
Either is OK I think.)
So one way to redo your macro would be like defstruct. In this case you would have one of
(some-macro some-name (...)
...)
or
(some-macro (some-name :flag) (...)
...)
And you could implement that pretty easily:
(defmacro some-macro (thing (&rest args) &body forms)
(multiple-value-bind (the-thing the-options)
(etypecase thing
(symbol (values thing '()))
(cons
(destructuring-bind (proposed-name . proposed-options) thing
(unless (symbolp proposed-name)
(error ...))
(unless (proper-list-p proposed-options)
(error ...))
(values proposed-name proposed-options))))
...))
In fact I would go further than this: people expect keyword arguments to have values, because in most other places they do. So instead have
(some-macro (some-name :flag t) (...)
...)
Which meets that expectation. This has the additional advantage that you can just use CL's argument parsing to get information:
> (destructuring-bind (&key (flag nil flagp)) '(:flag t)
(values flag flagp))
t
t
For instance. If you write the macro like this you might end up with something which looks like this:
(defmacro some-macro (thing (&rest args) &body forms)
(multiple-value-bind (the-thing flag flagp)
(etypecase thing
(symbol (values thing nil nil))
(cons
(destructuring-bind (proposed-name (&key (flag nil flagp))) thing
(unless (symbolp proposed-name)
(error ...))
(values proposed-name flag flagp))))
...))
As an aside it's worth considering why defclass & defstruct do things differently, and what this means for other macros.
defstruct looks, in outline, like
(defstruct structure-name-and-options
slot-description
...)
What this means is that, if you put options associated with the structure itself at the end, they're going to get confused with slot descriptions.
defclass gets around this by looking like this:
(defclass class-name (superclass-name ...)
(slot-description
...)
[class-option] ...)
It has nested the slot descriptions inside another list, which means that there is now space in the pattern for class options at the end of the form.
For macros which have some kind of 'body', then the natural pattern looks like
(with-foo something [some-more-special-things] ...
form
...)
For example
(with-slots (sa sb) x
(when (> sa sb)
(setf sb (+ sa sb)))
(values sa sb))
The problem here is that the whole tail of the macro form is the body which means that there is no natural place for options at the end: the only place to put them is at the beginning somewhere. You could get around this, again, by nesting the body:
(with-weird-thing x (y z)
((when y
...)
(print z)
...)
option ...)
But this, again, violates people's expectations: no (?) standard CL macros do this. It's significant that defclass's 'body' is not some forms: it's a list of slot specifications. So it's reasonable to adopt this pattern for defclass.
Finally it's worth considering defmethod. If I had designed this I would have done it slightly differently!
No-nonsense approach: rewrite the macro syntax with flags into another macro which takes a fixed argument containing the flags:
(defmacro some-macro-w-flags (flags name (&rest args) &body body)
...)
(defmacro some-macro (&rest args)
(let ((flags))
(loop while (keywordp (car args))
do (push (pop args) flags))
`(some-macro-w-flags ,flags ,#args)))
A few tests:
[1]> (macroexpand-1 '(some-macro abc (1 2 3)))
(SOME-MACRO-W-FLAGS NIL ABC (1 2 3)) ;
T
[2]> (macroexpand-1 '(some-macro :foo abc (1 2 3)))
(SOME-MACRO-W-FLAGS (:FOO) ABC (1 2 3)) ;
T
[3]> (macroexpand-1 '(some-macro :foo :bar abc (1 2 3)))
(SOME-MACRO-W-FLAGS (:BAR :FOO) ABC (1 2 3)) ;
T
I'm having a problem passing a list to macro, where the list will be used to generate function name. For example, the code below causes an error.
(defmacro gen (str-lst)
`(defun ,(intern (string-upcase (car str-lst))) () (print "foo")))
(gen '("foo" "bar"))
The resulting error was:
*** - DEFUN/DEFMACRO: QUOTE is a special operator and may not be redefined. The following restarts are available: ABORT :R1
Abort main loop
How should I modify my code, and what is wrong with my code?
The thing makes me even more confused is that the code below, about which answer exits here, works fine.
(defmacro easy-one (str-lst)
`(mapc #'(lambda (str) (print str)) ,str-lst))
(easy-one '("foo" "bar"))
Don't quote the list. Macros don't evaluate their arguments, so you don't need to quote them to prevent them from being evaluated, like you do for ordinary functions.
(gen ("foo" "bar"))
When you quote it, you're executing
(get (quote ("foo" "bar")))
The value of str-list is the list (quote ("foo" "bar")), so (car str-list) is the symbol QUOTE. As a result, the macro expands to
(defun quote () (print "foo"))
That's why you get an error complaining that you're trying to redefine the built-in QUOTE.
The difference in your second example is that you're just substituting the parameter into the expansion, you're not using its value in the expansion code. So it expands to
(mapc #'(lambda (str) (print str)) '("foo" "bar")))
Here the list will be used when the expansion runs, not while the macro is being expanded. It needs to be quoted there, to prevent it from being evaluated as a function call.
You should use macroexpand to see how your macros are being expanded when debugging.
Let's assume:
(defmacro testing (&optional var)
`(list 'this 'is
,#(when (consp var) `('a 'list))))
when called:
>(testing 2)
(THIS IS)
>(testing (list 1 2))
(THIS IS A LIST)
which is what I wanted. But now, when I pass a parameter that is a list:
>(defparameter bla (list 1 2 3))
BLA
>(testing bla)
(THIS IS)
which I suppose is because the macro would check (consp bla) where bla is a symbol, instead of the list? How do I prevent this?
Thanks
You could do something like:
(defmacro testing (&optional var)
`(if (consp ,var)
'(this is a list)
'(this is)))
So var will be evaluated at run time (not compile time). var only appears one time in the expansion of the macro, but if it appeared more than once, you would have to use a gensym.
EDIT: If you don't want to type '(this is) twice, do this:
(defmacro testing (&optional var)
`(append '(this is) (when (consp ,var) '(a list))))
Don't use eval, it's slow, and completely unnecessary. By substituting var into the macro expansion, it will naturally be evaluated at run-time. If you use eval, you will be doing something like this:
(eval (append '(list 'this 'is) (when (consp 'bla) '('a 'list))))
Every time that executes, it will build up a list representing the code and compile it before running it. (Hopefully this isn't in a loop!) If you just use a macro which generates straightforward code (without eval), it will compile only once.
The problem here is that the expression
,#(when (consp var) `('a 'list))))
is evaluated at compile time, when you only have literal (unevaluated) values of arguments. In your case: 2, (list 1 2), and bla.
The only solution to this, that I'm aware of, is to use eval. This particular example can be changed as follows:
(defmacro testing (&optional var)
`(eval (append '(list 'this 'is)
(when (consp ',var)
'('a 'list))))
But, I think, you'll agree, that its really ugly. And it won't work if you want to use lexical variables. Usually, there are ways to reformulate the problem, so that such perversions aren't needed.
I am working on a macro, I am trying to figure out how to avoid expansion of certain forms, take the following and macro for example,
(defmacro and
([] true)
([x] x)
([x & next]
`(let [and# ~x]
(if and# (and ~#next) and#))))
When expanded,
(mexpand-all '(and 1 2 3))
becomes,
(let* [and__973__auto__ 1]
(if and__973__auto__
(let* [and__973__auto__ 2]
(if and__973__auto__ 3 and__973__auto__))
and__973__auto__))
In this case what I need to do is stop let from expanding into let*.
Huh? It's not clear what you mean by "stop" let from expanding. let is a macro defined in clojure.core, which the compiler knows nothing about: it only understands let*. If your macro expanded into a let which (somehow) refused to expand further, it would fail to compile.
If you want to inspect just the output of your macro in isolation, without worrying about recursively expanding it, you should use macroexpand or macroexpand-1 instead of this mexpand-all thing. I don't know where mexpand-all comes from, but when I need something like that I use clojure.walk/macroexpand-all.
Recursive macro-expansion works by repeatedly expanding the form until there is no macro to expand. That means that if you want to recursively expand a macro, but ignore certain forms, you'll have to either code your own custom expander, or find someone else's.
Here's a quick example:
(defn my-expander [form]
(cond (not (list? form)) (mexpand-1 form)
(= (first form) 'let) form
:else (map my-expander (mexpand-1 form))))
Please forgive me if I made any mistakes. I'm much stronger with Scheme and CL than Clojure.
--Edit--
Note that the above function will not expand the subforms of a let statement, either.
Use macroexpand-1 to perform a single level of macro expansion.
After loading your and macro, this expression:
user=> (macroexpand-1 '(and 1 2 3))
Yields:
(clojure.core/let [and__1__auto__ 1] (if and__1__auto__ (clojure.core/and 2 3) and__1__auto__))