What is the best practice for selectively passing evaluated arguments to a macro form?
To elaborate: The usefulness of macros lies in its ability to receives unevaluated parameter, unlike the default evaluation rule for function forms. However, there is a legitimate use cases for evaluating macro arguments.
Consider a contrived example:
(defparameter *func-body* '((print i) (+ i 1)))
Suppose it would be nice that *func-body* could serve as the body of a macro our-defun that is defined as:
(defmacro our-defun (fun args &body body)
`(defun ,fun ,args ,#body))
So after (our-defun foo (i) (+ 1 i)), we could say (foo 1) to get 2. However, if we use (our-defun foo (i) *func-body*), the result of (foo 1) will be ((PRINT I) (+ I 1)) (i.e., the value of *func-body*). It would be nice if we can force the evaluation of *func-body* as an argument to the macro our-defun.
Currently, I can think of a technique of using compile and funcall to do this, as in
(funcall (compile nil `(lambda () (our-defun foo (i) ,#*func-body*))))
after which (our-defun 1) will print out 1 and return 2, as intended. I can think of case of making this work with eval, but I would rather stay away from eval due to its peculiarity in scoping.
This leads to my question at the begining, is there a more straightforward or native way to do this?
P.S.,
A not-so-contrived example is in the function (UPDATE-HOOK), which uses two library macros (ADD-HOOK) and (REMOVE-HOOK) and needs to evaluate its parameters. The (funcall (compile nil `(lambda () ...))) technique above is used here.
(defun update-hook (hook hook-name &optional code)
(funcall (compile nil `(lambda () (remove-hook ,hook ',hook-name))))
(unless (null code)
(compile hook-name `(lambda () ,#code))
(funcall (compile nil `(lambda () (add-hook ,hook ',hook-name))))))
That's slightly confused. A macro does not receive unevaluated parameters.
A macro gets source code and creates source code from that. Remember also that source code in Lisp is actually provided as data. The macro creates code, which evaluates some forms and some not.
Macros need to work in a compiling system. Before runtime. During compile time. All the macro sees is source code and then it creates source code from that. Think of macros as code transformations, not about evaluating arguments or not.
It would be nice if we can force the evaluation of *func-body* as an argument to the macro our-defun
That is not very clean. In a compiled system, you would need to make sure that *func-body* actually has a useful binding and that it can be resolved at COMPILE TIME.
If you have a macro like DEFUN, it makes sense to have the source code static. If you want to insert some source code into a form, then it could make sense to do that at read time:
(defun foo (i) #.`(,#*foo*))
But that's code I usually would want to avoid.
two library macros (ADD-HOOK) and (REMOVE-HOOK) and needs to evaluate its parameters.
Why should ADD-HOOK and REMOVE-HOOK be macros? If you don't have a real reason, they simply should be functions. Already since they make reuse difficult.
If you want to make ADD-HOOK and REMOVE-HOOK macros for some reason, then UPDATE-HOOK usually should be a macro, too.
The list you are giving to your macro has the form
(Quote (...))
So the list you actually want is the CADR of the list you get.
Related
I would like a more detailed explanation of how macro expansion works, at least in Emacs Lisp but an overview of other Lisps would be appreciated. The way I usually see it explained is that the arguments of the macro are passed unevaluated to the body, which is then executed and returns a new LISP form. However, if I do
(defun check-one (arg)
(eq arg 1))
(defmacro check-foo (checker foo)
(if (checker 1)
`(,foo "yes")
`(,foo "no")))
I would expect
(check-foo check-one print)
to first expand to
(if (check-one 1)
`(print "yes")
`(print "no))
and then finally to
(print "yes")
but instead I get a "checker" function is void error. On the other hand, if I had defined
(defmacro check-foo (checker foo)
(if (funcall checker 1)
`(,foo "yes")
`(,foo "no")))
then I would have the expected behavior. So the expressions do get replaced in the body unevaluated, but for some reason functions do not work? What is the step-by-step procedure the interpreter follows when macroexpanding? Is there a good text-book that explains this rigorously?
Macros are functions ...
A good way to think about macros is that they are simply functions, like any other function.
... which operate on source code
But they are functions whose arguments are source code, and whose value is also source code.
Looking at macro functions
Macros being functions is not quite explicit in elisp: some of the lower-level functionality is, I think, not exposed. But in Common Lisp this is quite literally how macros are implemented: a macro has an associated function, and this function gets called to expand the macro, with its value being the new source code. For instance, if you are so minded you could write macros in Common Lisp like this.
(defun expand-fn (form environment)
;; not talking about environment
(declare (ignore environment))
(let ((name (second form))
(arglist (third form))
(body (cdddr form)))
`(function (lambda ,arglist
(block ,name
,#body)))))
(setf (macro-function 'fn) #'expand-fn)
And now fn is a macro which will construct a function which 'knows its name', so you could write
(fn foo (x) ... (return-from foo x) ...)
which turns into
(function (lambda (x) (block foo ... (return-from foo x))))
In Common Lisp, defmacro is then itself a macro which arranges for a suitable macro function to be installed and also deals with making the macro available at compile time &c.
In elisp, it looks as if this lower layer is not specified by the language, but I think it's safe to assume that things work the same way.
So then the job of a macro is to take a bunch of source code and compute from it another bunch of source code which is the expansion of the macro. And of course the really neat trick is that, because source code (both arguments and values) is represented as s-expressions, Lisp is a superb language for manipulating s-expressions, you can write macros in Lisp itself.
Macroexpansion
There are a fair number of fiddly corner cases here such as local macros and so on. But here is, pretty much, how this works.
Start with some form <f>:
If <f> is (<a> ...) where <a> is a symbol, check for a macro function for <a>. If it has one, call it on the whole form, and call the value it returns <f'>: now simply recurse on <f'>.
If <f> is (<a> ...) where <a> is a symbol which names a special operator (something like if) then recurse on the subforms of the special operator which its rules say must be macroexpanded. As an example, in a form like (if <x> <y> <z>) all of <x>, <y>, & <z> need to be macroexpanded, while in (setq <a> <b>), only <b> would be subject to macroexpansion, and so on: these rules are hard-wired, which is why special operators are special.
If <f> is (<a> ...) where <a> is a symbol which is neither of the above cases, then it's a function call, and the forms in the body of the form are macroexpanded, and that's it.
If <f> is ((lambda (...) ...) ...) then the forms in the body of the lambda (but not its arguments!) are macroexpanded and then the case is the same as the last one.
Finally <f> might not be a compound form: nothing to do here.
I think that's all the cases. This is not a complete description of the process because there are complications like local macros and so on. But it's enough I think.
Order of macroexpansion
Note that macroexpansion happens 'outside in': a form like (a ...) is expanded until you get something which isn't a macro form, and only then is the body, perhaps, expanded. That's because, until the macro is completely expanded, you have no idea which, if any, of the subforms are even eligible for macroexpansion.
Your code
My guess is that what you want to happen is that (check-foo bog foo) should turn into (if (bog 1) (foo yes) (foo no)). So the way to get this is that this form is what the macro function needs to return. We could write this using the CL low-level facilities:
(defun check-foo-expander (form environment)
;; form is like (check-foo pred-name function-name)
(declare (ignore environment)) ;still not talking about environment
`(if (,(second form) 1)
(,(third form) "yes")
(,(third form) "no")))
And we can check:
> (check-foo-expander '(check-foo bog foo) nil)
(if (bog 1) (foo "yes") (foo "no"))
And then install it as a macro:
> (setf (macro-function 'check-foo) #'check-foo-expander)
And now
> (check-foo evenp print)
"no"
"no"
> (check-foo oddp print)
"yes"
"yes"
But it's easier to write it using defmacro:
(defmacro check-foo (predicate function)
`(if (,predicate 1)
(,function "yes")
(,function "no")))
This is the same thing (more-or-less), but easier to read.
In Lisp, a function's arguments are evaluated first before entering the function body. Macro arguments stay not evaluated.
But sometimes, one wants to inject code pieces stored in variables into a macro. This means evaluating the argument for the macro first, and then apply the macro-of-choice on this evaluated result.
One has to resort to
(eval `(macro ,arg))
To achieve this - but eval does not behave correctly in different environments.
The best thing would be, if one could do:
(apply macro (list arg))
or
(funcall macro arg)
But since the macro is not a function this doesn't work.
Is it possible to achieve something like this? - To circumvent that problem oder to make the macro available in the functions namespace?
Or am I missing some other ways to solve such problems?
I came to this question while trying to answer How to produce HTML from a list. but also in Generate TYPECASE with macro in common lisp, Evaluate arguments passed to a macro that generates functions in lisp, and How to convert a list to code/lambda in scheme?. But I always thought while answering them it would be good to have an apply or funcall-like function which can take macros.
It is not clear what you are trying to do, although it is almost certain that you are confused about something. In particular if you are calling eval inside macroexpansions then in almost all cases you are doing something both seriously wrong and seriously dangerous. I can't ever think of a case where I've wanted macros which expand to things including eval and I have written Lisp for a very very long time.
That being said, here is how you call the function associated with a macro, and why it is very seldom what you want to do.
Macros are simply functions whose domain and range is source code: they are compilers from a language to another language. It is perfectly possible to call the function associated with a macro, but what that function will return is source code, and what you will then need to do with that source code is evaluate it. If you want a function which deals with run-time data which is not source code, then you need that function, and you can't turn a macro into that function by some magic trick which seems to be what you want to do: that magic trick does not, and can not, exist.
So for instance if I have a macro
(defmacro with-x (&body forms)
`(let ((x 1))
,#forms))
Then I can call its macro function on a bit of source code:
> (funcall (macro-function 'with-x)
'(with-x (print "foo")) nil)
(let ((x 1)) (print "foo"))
But the result of this is another bit of source code: I need to compile or evaluate it, and nothing I can do will get around this.
Indeed in (almost?) all cases this is just the same as macroexpand-1):
> (macroexpand-1 '(with-x (print "foo")))
(let ((x 1)) (print "foo"))
t
And you can probably write macroexpand-1 in terms of macro-function:
(defun macroexpand-1/equivalent (form &optional (env nil))
(if (and (consp form)
(symbolp (first form))
(macro-function (first form)))
(values (funcall (macro-function (first form)) form env)
t)
(values form nil)))
So, if the result of calling a macro is source code, what do you do with that source code to get a result which is not source code? Well, you must evaluate it. And then, well, since the evaluator expands macros for you anyway, you might as well just write something like
(defun evaluate-with-x (code)
(funcall (compile nil `(lambda ()
(with-x ,#code)))))
So you didn't need to call the macro's function in any case. And this is not the magic trick which turns macros into functions dealing with data which is not source code: it is a terrible horror which is entirely made of exploding parts.
A concrete example: CL-WHO
It looks like this question might have its origins in this one and the underlying problem there is that that's not what CL-WHO does. In particular it is a confusion to think that something like CL-WHO is a tool for taking some kind of list and turning it into HTML. It's not: it's a tool for taking the source code of a language which is built on CL but includes a way of expressing HTML output mingled with CL code, and compiles it into CL code which will do the same thing. It happens to be the case that CL source code is expressed as lists & symbols, but CL-WHO isn't really about that: it's a compiler from, if you like, 'the CL-WHO language' to CL.
So, let's try the trick we tried above and see why it's a disaster:
(defun form->html/insane (form)
(funcall
(compile nil `(lambda ()
(with-html-output-to-string (,(make-symbol "O"))
,#form)))))
And you might, if you did not look at this too closely, think that this function does in fact do the magic trick:
> (form->html/insane '(:p ((:a :href "foo") "the foo")))
"<p></p><a href='foo'>the foo</a>"
But it doesn't. What happens if we call form->html/insane on this perfectly innocuous list:
(:p (uiop/run-program:run-program "rm -rf $HOME" :output t))
Hint: don't call form->html/insane on this list if you don't have very good backups.
CL-WHO is an implementation of a programming language which is a strict superset of CL: if you try to turn it into a function to turn lists into HTML you end up with something involving the same nuclear weapon you tinker with every time you call eval, except that nuclear weapon is hidden inside a locked cupboard where you can't see it. But it doesn't care about that: if you set it off it will still reduce everything within a few miles to radioactive ash and rubble.
So if you want a tool which will turn lists – lists which aren't source code – into HTML then write that tool. CL-WHO might have the guts of such a tool in its implemenentation, but you can't use it as it is.
And this is the same problem you face whenever you are trying to abuse macros this way: the result of calling a macro's function is Lisp source code, and to evaluate that source code you need eval or an equivalent of eval. And eval is not only not a terrible solution to almost any problem: it's also a nuclear weapon. There are, perhaps problems for which nuclear weapons are good solutions, but they are few and far between.
AND and OR are macros and since macros aren't first class in scheme/racket they cannot be passed as arguments to other functions. A partial solution is to use and-map or or-map. Is it possible to write a function that would take arbitrary macro and turn it into a function so that it can be passed as an argument to another function? Are there any languages that have first class macros?
In general, no. Consider that let is (or could be) implemented as a macro on top of lambda:
(let ((x 1))
(foo x))
could be a macro that expands to
((lambda (x) (foo x)) 1)
Now, what would it look like to convert let to a function? Clearly it is nonsense. What would its inputs be? Its return value?
Many macros will be like this. In fact, any macro that could be routinely turned into a function without losing any functionality is a bad macro! Such a macro should have been a function to begin with.
I agree with #amalloy. If something is written as a macro, it probably does something that functions can't do (e.g., introduce bindings, change evaluation order). So automatically converting arbitrary macro into a function is a really bad idea even if it is possible.
Is it possible to write a function that would take arbitrary macro and turn it into a function so that it can be passed as an argument to another function?
No, but it is somewhat doable to write a macro that would take some macro and turn it into a function.
#lang racket
(require (for-syntax racket/list))
(define-syntax (->proc stx)
(syntax-case stx ()
[(_ mac #:arity arity)
(with-syntax ([(args ...) (generate-temporaries (range (syntax-e #'arity)))])
#'(λ (args ...) (mac args ...)))]))
((->proc and #:arity 2) 42 12)
(apply (->proc and #:arity 2) '(#f 12))
((->proc and #:arity 2) #f (error 'not-short-circuit))
You might also be interested in identifier macro, which allows us to use an identifier as a macro in some context and function in another context. This could be used to create a first class and/or which short-circuits when it's used as a macro, but could be passed as a function value in non-transformer position.
On the topic of first class macro, take a look at https://en.wikipedia.org/wiki/Fexpr. It's known to be a bad idea.
Not in the way you probably expect
To see why, here is a way of thinking about macros: A macro is a function which takes a bit of source code and turns it into another bit of source code: the expansion of the macro. In other words a macro is a function whose domain and range are source code.
Once the source code is fully expanded, then it's fed to either an evaluator or a compiler. Let's assume it's fed to a compiler because it makes the question easier to answer: a compiler itself is simply a function whose domain is source code and whose range is some sequence of instructions for a machine (which may or may not be a real machine) to execute. Those instructions might include things like 'call this function on these arguments'.
So, what you are asking is: can the 'this function' in 'call this function on these arguments' be some kind of macro? Well, yes, it could be, but whatever source code it is going to transform certainly can not be the source code of the program you are executing, because that is gone: all that's left is the sequence of instructions that was the return value of the compiler.
So you might say: OK, let's say we disallow compilers: can we do it now? Well, leaving aside that 'disallowing compilers' is kind of a serious limitation, this was, in fact, something that very old dialects of Lisp sort-of did, using a construct called a FEXPR, as mentioned in another answer. It's important to realise that FEXPRs existed because people had not yet invented macros. Pretty soon, people did invent macros, and although FEXPRs and macros coexisted for a while – mostly because people had written code which used FEXPRs which they wanted to keep running, and because writing macros was a serious pain before things like backquote existed – FEXPRs died out. And they died out because they were semantically horrible: even by the standards of 1960s Lisps they were semantically horrible.
Here's one small example of why FEXPRs are so horrible: Let's say I write this function in a language with FEXPRs:
(define (foo f g x)
(apply f (g x)))
Now: what happens when I call foo? In particular, what happens if f might be a FEXPR?. Well, the answer is that I can't compile foo at all: I have to wait until run-time and make some on-the-fly decision about what to do.
Of course this isn't what these old Lisps with FEXPRs probably did: they would just silently have assumed that f was a normal function (which they would have called an EXPR) and compiled accordingly (and yes, even very old Lisps had compilers). If you passed something which was a FEXPR you just lost: either the thing detected that, or more likely it fall over horribly or gave you some junk answer.
And this kind of horribleness is why macros were invented: macros provide a semantically sane approach to processing Lisp code which allows (eventually, this took a long time to actually happen) minor details like compilation being possible at all, code having reasonable semantics and compiled code having the same semantics as interpreted code. These are features people like in their languages, it turns out.
Incidentally, in both Racket and Common Lisp, macros are explicitly functions. In Racket they are functions which operate on special 'syntax' objects because that's how you get hygiene, but in Common Lisp, which is much less hygienic, they're just functions which operate on CL source code, where the source code is simply made up of lists, symbols &c.
Here's an example of this in Racket:
> (define foo (syntax-rules ()
[(_ x) x]))
> foo
#<procedure:foo>
OK, foo is now just an ordinary function. But it's a function whose domain & range are Racket source code: it expects a syntax object as an argument and returns another one:
> (foo 1)
; ?: bad syntax
; in: 1
; [,bt for context]
This is because 1 is not a syntax object.
> (foo #'(x 1))
#<syntax:readline-input:5:10 1>
> (syntax-e (foo #'(x 1)))
1
And in CL this is even easier to see: Here's a macro definition:
(defmacro foo (form) form)
And now I can get hold of the macro's function and call it on some CL source code:
> (macro-function 'foo)
#<Function foo 4060000B6C>
> (funcall (macro-function 'foo) '(x 1) nil)
1
In both Racket and CL, macros are, in fact, first-class (or, in the case of Racket: almost first-class, I think): they are functions which operate on source code, which itself is first-class: you can write Racket and CL programs which construct and manipulate source code in arbitrary ways: that's what macros are in these languages.
In the case of Racket I have said 'almost first-class', because I can't see a way, in Racket, to retrieve the function which sits behind a macro defined with define-syntax &c.
I've created something like this in Scheme, it's macro that return lambda that use eval to execute the macro:
(define-macro (macron m)
(let ((x (gensym)))
`(lambda (,x)
(eval `(,',m ,#,x)))))
Example usage:
;; normal eval
(define x (map (lambda (x)
(eval `(lambda ,#x)))
'(((x) (display x)) ((y) (+ y y)))))
;; using macron macro
(define x (map (macron lambda)
'(((x) (display x)) ((y) (+ y y)))))
and x in both cases is list of two functions.
another example:
(define-macro (+++ . args)
`(+ ,#args))
((macron +++) '(1 2 3))
In this post, I ask tangentially why when I declare in SBCL
(defun a (&rest x)
x)
and then check what the function cell holds
(describe 'a)
COMMON-LISP-USER::A
[symbol]
A names a compiled function:
Lambda-list: (&REST X)
Derived type: (FUNCTION * (VALUES LIST &OPTIONAL))
Source form:
(LAMBDA (&REST X) (BLOCK A X))
I see this particular breakdown of the original function. Could someone explain what this output means? I'm especially confused by the last line
Source form:
(LAMBDA (&REST X) (BLOCK A X))
This is mysterious because for some reason not clear to me Lisp has transformed the original function into a lambda expression. It would also be nice to know the details of how a function broken down like this is then called. This example is SBCL. In Elisp
(symbol-function 'a)
gives
(lambda (&rest x) x)
again, bizarre. As I said in the other post, this is easier to understand in Scheme -- but that created confusion in the answers. So once more I ask, Why has Lisp taken a normal function declaration and seemingly stored it as a lambda expression?
I'm still a bit unclear what you are confused about, but here is an attempt to explain it. I will stick to CL (and mostly to ANSI CL), because elisp has a lot of historical oddities which just make things hard to understand (there is an appendix on elisp). Pre-ANSI CL was also a lot less clear on various things.
I'll try to explain things by writing a macro which is a simple version of defun: I'll call this defun/simple, and an example of its use will be
(defun/simple foo (x)
(+ x x))
So what I need to do is to work out what the expansion of this macro should be, so that it does something broadly equivalent (but simpler than) defun.
The function namespace & fdefinition
First of all I assume you are comfortable with the idea that, in CL (and elisp) the namespace of functions is different than the namespace of variable bindings: both languages are lisp-2s. So in a form like (f x), f is looked up in the namespace of function bindings, while x is looked up in the namespace of variable bindings. This means that forms like
(let ((sin 0.0))
(sin sin))
are fine in CL or elisp, while in Scheme they would be an error, as 0.0 is not a function, because Scheme is a lisp-1.
So we need some way of accessing that namespace, and in CL the most general way of doing that is fdefinition: (fdefinition <function name>) gets the function definition of <function name>, where <function name> is something which names a function, which for our purposes will be a symbol.
fdefinition is what CL calls an accessor: this means that the setf macro knows what to do with it, so that we can mutate the function binding of a symbol by (setf (fdefinition ...) ...). (This is not true: what we can access and mutate with fdefinition is the top-level function binding of a symbol, we can't access or mutate lexical function bindings, and CL provides no way to do this, but this does not matter here.)
So this tells us what our macro expansion needs to look like: we want to set the (top-level) definition of the name to some function object. The expansion of the macro should be like this:
(defun/simple foo (x)
x)
should expand to something involving
(setf (fdefinition 'foo) <form which makes a function>)
So we can write this bit of the macro now:
(defmacro defun/simple (name arglist &body forms)
`(progn
(setf (fdefinition ',name)
,(make-function-form name arglist forms))
',name))
This is the complete definition of this macro. It uses progn in its expansion so that the result of expanding it is the name of the function being defined, which is the same as defun: the expansion does all its real work by side-effect.
But defun/simple relies on a helper function, called make-function-form, which I haven't defined yet, so you can't actually use it yet.
Function forms
So now we need to write make-function-form. This function is called at macroexpansion time: it's job is not to make a function: it's to return a bit of source code which will make a function, which I'm calling a 'function form'.
So, what do function forms look like in CL? Well, there's really only one such form in portable CL (this might be wrong, but I think it is true), which is a form constructed using the special operator function. So we're going to need to return some form which looks like (function ...). Well, what can ... be? There are two cases for function.
(function <name>) denotes the function named by <name> in the current lexical environment. So (function car) is the function we call when we say (car x).
(function (lambda ...)) denotes a function specified by (lambda ...): a lambda expression.
The second of these is the only (caveats as above) way we can construct a form which denotes a new function. So make-function-form is going to need to return this second variety of function form.
So we can write an initial version of make-function-form:
(defun make-function-form (name arglist forms)
(declare (ignore name))
`(function (lambda ,arglist ,#forms)))
And this is enough for defun/simple to work:
> (defun/simple plus/2 (a b)
(+ a b))
plus/2
> (plus/2 1 2)
3
But it's not quite right yet: one of the things that functions defined by defun can do is return from themselves: they know their own name and can use return-from to return from it:
> (defun silly (x)
(return-from silly 3)
(explode-the-world x))
silly
> (silly 'yes)
3
defun/simple can't do this, yet. To do this, make-function-form needs to insert a suitable block around the body of the function:
(defun make-function-form (name arglist forms)
`(function (lambda ,arglist
(block ,name
,#forms))))
And now:
> (defun/simple silly (x)
(return-from silly 3)
(explode-the-world x))
silly
> (silly 'yes)
3
And all is well.
This is the final definition of defun/simple and its auxiliary function.
Looking at the expansion of defun/simple
We can do this with macroexpand in the usual way:
> (macroexpand '(defun/simple foo (x) x))
(progn
(setf (fdefinition 'foo)
#'(lambda (x)
(block foo
x)))
'foo)
t
The only thing that's confusing here is that, because (function ...) is common in source code, there's syntactic sugar for it which is #'...: this is the same reason that quote has special syntax.
It's worth looking at the macroexpansion of real defun forms: they usually have a bunch of implementation-specific stuff in them, but you can find the same thing there. Here's an example from LW:
> (macroexpand '(defun foo (x) x))
(compiler-let ((dspec::*location* '(:inside (defun foo) :listener)))
(compiler::top-level-form-name (defun foo)
(dspec:install-defun 'foo
(dspec:location)
#'(lambda (x)
(declare (system::source-level
#<eq Hash Table{0} 42101FCD5B>))
(declare (lambda-name foo))
x))))
t
Well, there's a lot of extra stuff in here, and LW obviously has some trick around this (declare (lambda-name ...)) form which lets return-from work without an explicit block. But you can see that basically the same thing is going on.
Conclusion: how you make functions
In conclusion: a macro like defun, or any other function-defining form, needs to expand to a form which, when evaluated, will construct a function. CL offers exactly one such form: (function (lambda ...)): that's how you make functions in CL. So something like defun necessarily has to expand to something like this. (To be precise: any portable version of defun: implementations are somewhat free to do implementation-magic & may do so. However they are not free to add a new special operator.)
What you are seeing when you call describe is that, after SBCL has compiled your function, it's remembered what the source form was, and the source form was exactly the one you would have got from the defun/simple macro given here.
Notes
lambda as a macro
In ANSI CL, lambda is defined as a macro whose expansion is a suitable (function (lambda ...)) form:
> (macroexpand '(lambda (x) x))
#'(lambda (x) x)
t
> (car (macroexpand '(lambda (x) x)))
function
This means that you don't have to write (function (lambda ...)) yourself: you can rely on the macro definition of lambda doing it for you. Historically, lambda wasn't always a macro in CL: I can't find my copy of CLtL1, but I'm pretty certain it was not defined as one there. I'm reasonably sure that the macro definition of lambda arrived so that it was possible to write ISLisp-compatible programs on top of CL. It has to be in the language because lambda is in the CL package and so users can't portably define macros for it (although quite often they did define such a macro, or at least I did). I have not relied on this macro definition above.
defun/simple does not purport to be a proper clone of defun: its only purpose is to show how such a macro can be written. In particular it doesn't deal with declarations properly, I think: they need to be lifted out of the block & are not.
Elisp
Elisp is much more horrible than CL. In particular, in CL there is a well-defined function type, which is disjoint from lists:
> (typep '(lambda ()) 'function)
nil
> (typep '(lambda ()) 'list)
t
> (typep (function (lambda ())) 'function)
t
> (typep (function (lambda ())) 'list)
nil
(Note in particular that (function (lambda ())) is a function, not a list: function is doing its job of making a function.)
In elisp, however, an interpreted function is just a list whose car is lambda (caveat: if lexical binding is on this is not the case: it's then a list whose car is closure). So in elisp (without lexical binding):
ELISP> (function (lambda (x) x))
(lambda (x)
x)
And
ELISP> (defun foo (x) x)
foo
ELISP> (symbol-function 'foo)
(lambda (x)
x)
The elisp intepreter then just interprets this list, in just the way you could yourself. function in elisp is almost the same thing as quote.
But function isn't quite the same as quote in elisp: the byte-compiler knows that, when it comes across a form like (function (lambda ...)) that this is a function form, and it should byte-compile the body. So, we can look at the expansion of defun in elisp:
ELISP> (macroexpand '(defun foo (x) x))
(defalias 'foo
#'(lambda (x)
x))
(It turns out that defalias is the primitive thing now.)
But if I put this definition in a file, which I byte compile and load, then:
ELISP> (symbol-function 'foo)
#[(x)
"\207"
[x]
1]
And you can explore this a bit further: if you put this in a file:
(fset 'foo '(lambda (x) x))
and then byte compile and load that, then
ELISP> (symbol-function 'foo)
(lambda (x)
x)
So the byte compiler didn't do anything with foo because it didn't get the hint that it should. But foo is still a fine function:
ELISP> (foo 1)
1 (#o1, #x1, ?\C-a)
It just isn't compiled. This is also why, if writing elisp code with anonymous functions in it, you should use function (or equivalently #'). (And finally, of course, (function ...) does the right thing if lexical scoping is on.)
Other ways of making functions in CL
Finally, I've said above that function & specifically (function (lambda ...)) is the only primitive way to make new functions in CL. I'm not completely sure that's true, especially given CLOS (almost any CLOS will have some kind of class instances of which are functions but which can be subclassed). But it does not matter: it is a way and that's sufficient.
DEFUN is a defining macro. Macros transform code.
In Common Lisp:
(defun foo (a)
(+ a 42))
Above is a definition form, but it will be transformed by DEFUN into some other code.
The effect is similar to
(setf (symbol-function 'foo)
(lambda (a)
(block foo
(+ a 42))))
Above sets the function cell of the symbol FOO to a function. The BLOCK construct is added by SBCL, since in Common Lisp named functions defined by DEFUN create a BLOCK with the same name as the function name. This block name can then be used by RETURN-FROM to enable a non-local return from a specific function.
Additionally DEFUN does implementation specific things. Implementations also record development information: the source code, the location of the definition, etc.
Scheme has DEFINE:
(define (foo a)
(+ a 10))
This will set FOO to a function object.
In Common Lisp, a macro definition must have been seen before the first use. This allows a macro to refer to itself, but does not allow two macros to refer to each other. The restriction is slightly awkward, but understandable; it makes the macro system quite a bit easier to implement, and to understand how the implementation works.
Is there any Lisp family language in which two macros can refer to each other?
What is a macro?
A macro is just a function which is called on code rather than data.
E.g., when you write
(defmacro report (x)
(let ((var (gensym "REPORT-")))
`(let ((,var ,x))
(format t "~&~S=<~S>~%" ',x ,var)
,var)))
you are actually defining a function which looks something like
(defun macro-report (system::<macro-form> system::<env-arg>)
(declare (cons system::<macro-form>))
(declare (ignore system::<env-arg>))
(if (not (system::list-length-in-bounds-p system::<macro-form> 2 2 nil))
(system::macro-call-error system::<macro-form>)
(let* ((x (cadr system::<macro-form>)))
(block report
(let ((var (gensym "REPORT-")))
`(let ((,var ,x)) (format t "~&~s=<~s>~%" ',x ,var) ,var))))))
I.e., when you write, say,
(report (! 12))
lisp actually passes the form (! 12) as the 1st argument to macro-report which transforms it into:
(LET ((#:REPORT-2836 (! 12)))
(FORMAT T "~&~S=<~S>~%" '(! 12) #:REPORT-2836)
#:REPORT-2836)
and only then evaluates it to print (! 12)=<479001600> and return 479001600.
Recursion in macros
There is a difference whether a macro calls itself in implementation or in expansion.
E.g., a possible implementation of the macro and is:
(defmacro my-and (&rest args)
(cond ((null args) T)
((null (cdr args)) (car args))
(t
`(if ,(car args)
(my-and ,#(cdr args))
nil))))
Note that it may expand into itself:
(macroexpand '(my-and x y z))
==> (IF X (MY-AND Y Z) NIL) ; T
As you can see, the macroexpansion contains the macro being defined.
This is not a problem, e.g., (my-and 1 2 3) correctly evaluates to 3.
However, if we try to implement a macro using itself, e.g.,
(defmacro bad-macro (code)
(1+ (bad-macro code)))
you will get an error (a stack overflow or undefined function or ...) when you try to use it, depending on the implementation.
Here's why mutually recursive macros can't work in any useful way.
Consider what a system which wants to evaluate (or compile) Lisp code for a slightly simpler Lisp than CL (so I'm avoiding some of the subtleties that happen in CL), such as the definition of a function, needs to do. It has a very small number of things it knows how to do:
it knows how to call functions;
it knows how to evaluate a few sorts of literal objects;
it has some special rules for a few sorts of forms – what CL calls 'special forms', which (again in CL-speak) are forms whose car is a special operator;
finally it knows how to look to see whether forms correspond to functions which it can call to transform the code it is trying to evaluate or compile – some of these functions are predefined but additional ones can be defined.
So the way the evaluator works is by walking over the thing it needs to evaluate looking for these source-code-transforming things, aka macros (the last case), calling their functions and then recursing on the results until it ends up with code which has none left. What's left should consist only of instances of the first three cases, which it then knows how to deal with.
So now think about what the evaluator has to do if it is evaluating the definition of the function corresponding to a macro, called a. In Cl-speak it is evaluating or compiling a's macro function (which you can get at via (macro-function 'a) in CL). Let's assume that at some point there is a form (b ...) in this code, and that b is known also to correspond to a macro.
So at some point it comes to (b ...), and it knows that in order to do this it needs to call b's macro function. It binds suitable arguments and now it needs to evaluate the definition of the body of that function ...
... and when it does this it comes across an expression like (a ...). What should it do? It needs to call a's macro function, but it can't, because it doesn't yet know what it is, because it's in the middle of working that out: it could start trying to work it out again, but this is just a loop: it's not going to get anywhere where it hasn't already been.
Well, there's a horrible trick you could do to avoid this. The infinite regress above happens because the evaluator is trying to expand all of the macros ahead of time, and so there's no base to the recursion. But let's assume that the definition of a's macro function has code which looks like this:
(if <something>
(b ...)
<something not involving b>)
Rather than doing the expand-all-the-macros-first trick, what you could do is to expand only the macros you need, just before you need their results. And if <something> turned out always to be false, then you never need to expand (b ...), so you never get into this vicious loop: the recursion bottoms out.
But this means you must always expand macros on demand: you can never do it ahead of time, and because macros expand to source code you can never compile. In other words a strategy like this is not compatible with compilation. It also means that if <something> ever turns out to be true then you'll end up in the infinite regress again.
Note that this is completely different to macros which expand to code which involves the same macro, or another macro which expands into code which uses it. Here's a definition of a macro called et which does that (it doesn't need to do this of course, this is just to see it happen):
(defmacro et (&rest forms)
(if (null forms)
't
`(et1 ,(first forms) ,(rest forms))))
(defmacro et1 (form more)
(let ((rn (make-symbol "R")))
`(let ((,rn ,form))
(if ,rn
,rn
(et ,#more)))))
Now (et a b c) expands to (et1 a (b c)) which expands to (let ((#:r a)) (if #:r #:r (et b c))) (where all the uninterned things are the same thing) and so on until you get
(let ((#:r a))
(if #:r
#:r
(let ((#:r b))
(if #:r
#:r
(let ((#:r c))
(if #:r
#:r
t))))))
Where now not all the uninterned symbols are the same
And with a plausible macro for let (let is in fact a special operator in CL) this can get turned even further into
((lambda (#:r)
(if #:r
#:r
((lambda (#:r)
(if #:r
#:r
((lambda (#:r)
(if #:r
#:r
t))
c)))
b)))
a)
And this is an example of 'things the system knows how to deal with': all that's left here is variables, lambda, a primitive conditional and function calls.
One of the nice things about CL is that, although there is a lot of useful sugar, you can still poke around in the guts of things if you like. And in particular, you still see that macros are just functions that transform source code. The following does exactly what the defmacro versions do (not quite: defmacro does the necessary cleverness to make sure the macros are available early enough: I'd need to use eval-when to do that with the below):
(setf (macro-function 'et)
(lambda (expression environment)
(declare (ignore environment))
(let ((forms (rest expression)))
(if (null forms)
't
`(et1 ,(first forms) ,(rest forms))))))
(setf (macro-function 'et1)
(lambda (expression environment)
(declare (ignore environment))
(destructuring-bind (_ form more) expression
(declare (ignore _))
(let ((rn (make-symbol "R")))
`(let ((,rn ,form))
(if ,rn
,rn
(et ,#more)))))))
There have been historic Lisp systems that allow this, at least in interpreted code.
We can allow a macro to use itself for its own definition, or two or more macros to mutually use each other, if we follow an extremely late expansion strategy.
That is to say, our macro system expands a macro call just before it is evaluated (and does that each time that same expression is evaluated).
(Such a macro expansion strategy is good for interactive development with macros. If you fix a buggy macro, then all code depending on it automatically benefits from the change, without having to be re-processed in any way.)
Under such a macro system, suppose we have a conditional like this:
(if (condition)
(macro1 ...)
(macro2 ...))
When (condition) is evaluated, then if it yields true, (macro1 ...) is evaluated, otherwise (macro2 ...). But evaluation also means expansion. Thus only one of these two macros is expanded.
This is the key to why mutual references among macros can work: we are able rely on the conditional logic to give us not only conditional evaluation, but conditional expansion also, which then allows the recursion to have ways of terminating.
For example, suppose macro A's body of code is defined with the help of macro B, and vice versa. And when a particular invocation of A is executed, it happens to hit the particular case that requires B, and so that B call is expanded by invocation of macro B. B also hits the code case that depends on A, and so it recurses into A to obtain the needed expansion. But, this time, A is called in a way that avoids requiring, again, an expansion of B; it avoids evaluating any sub-expression containing the B macro. Thus, it calculates the expansion, and returns it to B, which then calculates its expansion returns to the outermost A. A finally expands and the recursion terminates; all is well.
What blocks macros from using each other is the unconditional expansion strategy: the strategy of fully expanding entire top-level forms after they are read, so that the definitions of functions and macros contain only expanded code. In that situation there is no possibility of conditional expansion that would allow for the recursion to terminate.
Note, by the way, that a macro system which expands late doesn't recursively expand macros in a macro expansion. Suppose (mac1 x y) expands into (if x (mac2 y) (mac3 y)). Well, that's all the expansion that is done for now: the if that pops out is not a macro, so expansion stops, and evaluation proceeds. If x yields true, then mac2 is expanded, and mac3 is not.