I currently have a macro defined as follows:
(defmacro some-macro (generic-name (&rest args) &body body)
...)
I would now like to add an extra parameter to this macro, a kind of flag which a user can optionally provide and which would alter the behavior of the macro. The below code should be possible:
(some-macro some-name (arg1 arg2) (print (+ arg1 arg2)))
(some-macro :flag some-name (arg1 arg2) (print (special-+ arg1 arg2)))
I intentionally just made a conceptual example to focus on what's important instead of the real macro I'm trying to implement. The real a macro is different and more complex. As you can see, the macro should now be callable with both the :flag argument (flag can be any word, as long as its prefix is ':') and without the flag argument. Is there a way to do this without positioning the &optional keyword at the end of the parameter list (ie. it really needs to be at that first position).
&optional can only be at the end of the positional arguments (you can have &rest or &body after it). Even if you could put it earlier, if there's also &rest or &body, how would it know whether you provided the optional argument? E.g. if the lambda list were
(&optional arg1 arg2 &rest rest-args)
and the call were
(func-name 1 2 3 4 5)
it could either be arg1 = 1, arg2 = 2, rest-args = (3 4 5) or arg1 = NIL, arg2 = 1, rest-args = (2 3 4 5).
You'll need to define the macro to take a single &rest argument. Then you can check whether the first argument is a keyword or not, update the argument list to add the default, and then parse it with destructuring-bind.
(defmacro some-macro (&rest all-args)
(unless (keywordp (first all-args))
(push nil all-args)) ;; flag defaults to NIL
(destructuring-bind (flag generic-name (&rest args) &body body) all-args
...))
(some-macro :flag some-name (arg1 arg2) (print (special-+ arg1 arg2)))
(some-macro some-name (arg1 arg2) (print (+ arg1 arg2)))
One can only guess what's useful, since the design of the macro depends a bit on more context: what is it actually used for.
For example in CLOS we write
(defmethod foo :around ((a class-a)) ...)
The name comes first, then zero or more method-qualifiers (here :around and then the arglist. Putting a flag in front of a name would be strange in typical defining macros (those who begin with def).
For that we would need to write the macro-destructuring of the arglist ourselves, since it does not match a built-in macro arglist pattern.
(defmacro some-macro (name &rest args)
(let* ((qualifiers (loop for arg in args
until (listp arg)
collect (pop args)))
(arg-list (pop args))
(body args))
... ; return something for the example
))
In other macros we might write
(some-macro some-name (arg1 arg2 :flag)
(print (special-+ arg1 arg2)))
(defmacro some-macro (some-name (arg1 arg2 &optional flag) &body body)
...)
Similar for example to
(with-input-from-string (stream string :start 10)
(... ))
Though above uses keywords, not optionals.
Or we might want to write:
(some-macro some-name (arg1 arg2) (:flag1)
(print (special-+ arg1 arg2)))
(defmacro some-macro (some-name (&rest args) (&rest flags) &body body)
...)
If there are only three flags one can also generate three different macros with different names and remove the flag.
One thing to think about when designing things like macros (and, remember, when you are designing macros, you are designing a programming language) is how people expect to read the language. If the programming language you are designing is a mild superset of CL, or is otherwise very close to CL, then you may want to not violate the expectations CL programmers have when reading CL code, or more generally that Lisp programmers have when reading Lisp code.
[Note that most of what follows is opinion: people obviously have different opinions – these are mine, they're not more right than anyone else's.]
So, what are those expectations? Well, they may include the following two:
people read CL left-to-right, so the things at the left-hand end of a form tend to be more visually important;
many existing forms in CL look like (<operator> <thing> ...) – the first two subforms in the form are by far the most interesting, and the second is often more interesting than the first. Think about (defun foo ...), (dolist (x ...) ...), (let ((x y) ...) ...).
An example of something which violates these expectations is an object system which works explicitly with message-passing, using some send operation (I think Old Flavors did this, and I think that New Flavors didn't, but my memory is vague now). Code written using these looks like (send <object> <message> ...): the first word in many forms is send. That means that this visually-important place for reading code has been entirely wasted as it is always the same word and the important places are now the second and third subform. Well, instead, we could just omit the whole send thing and write (message object ...) or (object message ...). CLOS takes essentially the first of these options, where a 'message' is a generic function and of course generic functions can specialise on more than one argument which breaks the whole message-passing paradigm. But you can write CLOS as if it was message-passing, and it works, and it means that it agrees with how a lot of other CL code looks.
So, OK let's look at two cases of your macro:
(some-macro some-name ...)
This is fine.
(some-macro :flag some-name ...)
But this has filled the visual second-position slot with something which is not whatever the macro is about: it's just some optional argument. The interesting thing is now the third position.
Well, how could we fix this? It turns out there is a good example which already exists in CL: defstruct. defstruct has two basic cases:
(defstruct structure-name
...)
and
(defstruct (structure-name ...)
...)
Both of these meet the first-two-positions-matter-most requirement, while allowing optional arguments and clearly visually indicating when they are used.
(Aside: defclasss does things differently by putting the options at the end as in:
(defclass name (...supers...)
(...slot specifications...)
...options...))
Either is OK I think.)
So one way to redo your macro would be like defstruct. In this case you would have one of
(some-macro some-name (...)
...)
or
(some-macro (some-name :flag) (...)
...)
And you could implement that pretty easily:
(defmacro some-macro (thing (&rest args) &body forms)
(multiple-value-bind (the-thing the-options)
(etypecase thing
(symbol (values thing '()))
(cons
(destructuring-bind (proposed-name . proposed-options) thing
(unless (symbolp proposed-name)
(error ...))
(unless (proper-list-p proposed-options)
(error ...))
(values proposed-name proposed-options))))
...))
In fact I would go further than this: people expect keyword arguments to have values, because in most other places they do. So instead have
(some-macro (some-name :flag t) (...)
...)
Which meets that expectation. This has the additional advantage that you can just use CL's argument parsing to get information:
> (destructuring-bind (&key (flag nil flagp)) '(:flag t)
(values flag flagp))
t
t
For instance. If you write the macro like this you might end up with something which looks like this:
(defmacro some-macro (thing (&rest args) &body forms)
(multiple-value-bind (the-thing flag flagp)
(etypecase thing
(symbol (values thing nil nil))
(cons
(destructuring-bind (proposed-name (&key (flag nil flagp))) thing
(unless (symbolp proposed-name)
(error ...))
(values proposed-name flag flagp))))
...))
As an aside it's worth considering why defclass & defstruct do things differently, and what this means for other macros.
defstruct looks, in outline, like
(defstruct structure-name-and-options
slot-description
...)
What this means is that, if you put options associated with the structure itself at the end, they're going to get confused with slot descriptions.
defclass gets around this by looking like this:
(defclass class-name (superclass-name ...)
(slot-description
...)
[class-option] ...)
It has nested the slot descriptions inside another list, which means that there is now space in the pattern for class options at the end of the form.
For macros which have some kind of 'body', then the natural pattern looks like
(with-foo something [some-more-special-things] ...
form
...)
For example
(with-slots (sa sb) x
(when (> sa sb)
(setf sb (+ sa sb)))
(values sa sb))
The problem here is that the whole tail of the macro form is the body which means that there is no natural place for options at the end: the only place to put them is at the beginning somewhere. You could get around this, again, by nesting the body:
(with-weird-thing x (y z)
((when y
...)
(print z)
...)
option ...)
But this, again, violates people's expectations: no (?) standard CL macros do this. It's significant that defclass's 'body' is not some forms: it's a list of slot specifications. So it's reasonable to adopt this pattern for defclass.
Finally it's worth considering defmethod. If I had designed this I would have done it slightly differently!
No-nonsense approach: rewrite the macro syntax with flags into another macro which takes a fixed argument containing the flags:
(defmacro some-macro-w-flags (flags name (&rest args) &body body)
...)
(defmacro some-macro (&rest args)
(let ((flags))
(loop while (keywordp (car args))
do (push (pop args) flags))
`(some-macro-w-flags ,flags ,#args)))
A few tests:
[1]> (macroexpand-1 '(some-macro abc (1 2 3)))
(SOME-MACRO-W-FLAGS NIL ABC (1 2 3)) ;
T
[2]> (macroexpand-1 '(some-macro :foo abc (1 2 3)))
(SOME-MACRO-W-FLAGS (:FOO) ABC (1 2 3)) ;
T
[3]> (macroexpand-1 '(some-macro :foo :bar abc (1 2 3)))
(SOME-MACRO-W-FLAGS (:BAR :FOO) ABC (1 2 3)) ;
T
Related
Short version:
I want to change the #+ and #- reader macros to apply to all immediately subsequent tokens starting with ##, in addition to the following token. Therefore, the following code...
#+somefeature
##someattribute1
##someattribute2
(defun ...)
...would, in the absence of somefeature, result in no code.
Long version:
I have written my own readtable-macros which apply transformations to subsequent code. For example:
##traced
(defun ...)
This yields a function that writes its arguments and return values to a file, for debugging.
This fails, however, when used in conjunction with the #+ reader macro:
#+somefeature
##traced
(defun ...)
In the absence of somefeature, the function continues to be defined, albeit without the ##traced modification. This is obviously not the desired outcome.
One possible solution would be to use progn, as follows:
#+somefeature
(progn
##traced
(defun ...))
But that's kind of ugly.
I would like to modify the #+ and #- reader macros, such that they may consume more than one token. Something like this:
(defun conditional-syntax-reader (stream subchar arg)
; If the conditional fails, consume subsequent tokens while they
; start with ##, then consume the next token.
)
(setf *readtable* (copy-readtable))
(set-dispatch-macro-character #\# #\+ #'conditional-syntax-reader)
(set-dispatch-macro-character #\# #\- #'conditional-syntax-reader)
The problem is, I don't know how to "delegate" to the original reader macros; and I don't understand enough about how they were implemented to re-implement them myself in their entirety.
A naive approach would be:
(defun consume-tokens-recursively (stream)
(let ((token (read stream t nil t)))
(when (string= "##" (subseq (symbol-string token) 0 2))
(consume-tokens-recursively stream)))) ; recurse
(defun conditional-syntax-reader (stream subchar arg)
(unless (member (read stream t nil t) *features*)
(consume-tokens-recursively stream)))
However, I'm given to believe that this wouldn't be sufficient:
The #+ syntax operates by first reading the feature specification and then skipping over the form if the feature is false. This skipping of a form is a bit tricky because of the possibility of user-defined macro characters and side effects caused by the #. and #, constructions. It is accomplished by binding the variable read-suppress to a non-nil value and then calling the read function.
This seems to imply that I can just let ((*read-suppress* t)) when using read to solve the issue. Is that right?
EDIT 1
Upon further analysis, it seems the problem is caused by not knowing how many tokens to consume. Consider the following attributes:
##export expects one argument: the (defun ...) to export.
##traced expects two arguments: the debug level and the (defun ...) to trace.
Example:
#+somefeature
##export
##traced 3
(defun ...)
It turns out that #+ and #- are capable of suppressing all these tokens; but there is a huge problem!
When under a suppressing #+ or #-, (read) returns NIL!
Example:
(defun annotation-syntax-reader (stream subchar arg)
(case (read stream t nil t)
('export
(let ((defun-form (read stream t nil t)))))
; do something
('traced
(let* ((debug-level (read stream t nil t))
(defun-form (read stream t nil t)))))))
; do something
(setf *readtable* (copy-readtable))
(set-dispatch-macro-character #\# #\# #'annotation-syntax-reader)
#+(or) ##traced 3 (defun ...)
The ##traced token is being suppressed by the #+. In this situation, all the (read) calls in (annotation-syntax-reader) consume real tokens but return NIL!
Therefore, the traced token is consumed, but the case fails. No additional tokens are thus consumed; and control leaves the scope of the #+.
The (defun ...) clause is executed as normal, and the function comes into being. Clearly not the desired outcome.
The standard readtable
Changing the macros for #+ and #- is a bit excessive solution I think, but in any case remember to not actually change the standard readtable (as you did, but its important to repeat in the answer)
The consequences are undefined if an attempt is made to modify the standard readtable. To achieve the effect of altering or extending standard syntax, a copy of the standard readtable can be created; see the function copy-readtable.
§2.1.1.2 The Standard Readtable
Now, maybe I'm missing something (please give us a hint about how your reader macro is defined if so), but I think it is possible to avoid that and write your custom macros in a way that works for your use case.
Reader macro
Let's define a simple macro as follows:
CL-USER> (defun my-reader (stream char)
(declare (ignore char))
(let ((name (read stream)
(form (read stream))
(unless *read-suppress*
`(with-decoration ,name ,form)))
MY-READER
[NB: This was edited to take into account *read-suppress*: the code always read two forms, but returns nil in case it is being ignored. In the comments you say that you may need to read an indefinite number of forms based on the name of the decoration, but with *read-suppress* the recursive calls to read return nil for symbols, so you don't know which decoration is being applied. In that case it might be better to wrap some arguments in a literal list, or parse the stream manually (read-char, etc.). Also, since you are using a dispatching macro, maybe you can add a numerical argument if you want the decoration to be applied to more than one form (#2#inline), but that could be a bad idea when later the decorated code is being modified.]
Here the reader does a minimal job, namely build a form that is intended to be macroexpanded later. I don't even need to define with-decoration for now, as I'm interested in the read step. The intent is to read the next token (presumably a symbol that indicates what decoration is being applied, and a form to decorate).
I'm binding this macro to a unused character:
CL-USER> (set-macro-character #\§ 'my-reader)
T
Here when I test the macro it wraps the following form:
CL-USER> (read-from-string "§test (defun)")
(WITH-DECORATION TEST (DEFUN))
13 (4 bits, #xD, #o15, #b1101)
And here it works with a preceding QUOTE too, the apostrophe reader grabs the next form, which recursively reads two forms:
CL-USER> '§test (defun)
(WITH-DECORATION TEST (DEFUN))
Likewise, a conditional reader macro will ignore all the next lines:
CL-USER> #+(or) t
; No values
CL-USER> #+(or) §test (defun)
; No values
CL-USER> #+(or) §one §two §three (defun)
; No values
Decoration macro
If you use this syntax, you'll have nested decorated forms:
CL-USER> '§one §two (defun test ())
(WITH-DECORATION ONE (WITH-DECORATION TWO (DEFUN TEST ())))
With respect to defun in toplevel positions, you can arrange for your macros to unwrap the nesting (not completely tested, there might be bugs):
(defun unwrap-decorations (form stack)
(etypecase form
(cons (destructuring-bind (head . tail) form
(case head
(with-decoration (destructuring-bind (token form) tail
(unwrap-decorations form (cons token stack))))
(t `(with-decorations ,(reverse stack) ,form)))))))
CL-USER> (unwrap-decorations ** nil)
(WITH-DECORATIONS (ONE TWO) (DEFUN TEST ()))
And in turn, with-decorations might know about DEFUN forms and how to annotate them as necessary.
For the moment, our original macro is only the following (it needs more error checking):
(defmacro with-decoration (&whole whole &rest args)
(unwrap-decorations whole nil))
For the sake of our example, let's define a generic annotation mechanism:
CL-USER> (defgeneric expand-decoration (type name rest))
#<STANDARD-GENERIC-FUNCTION COMMON-LISP-USER::EXPAND-DECORATION (0)>
It is used in with-decorations to dispatch on an appropriate expander for each decoration. Keep in mind that all the efforts here are to keep defun in a top-level positions (under a progn), a recursive annotation would let evaluation happens (in the case of defun, it would result in the name of the function being defined), and the annotation could be done on the result.
The main macro is then here, with a kind of fold (reduce) mechanism where the forms are decorated using the resulting expansion so far. This allows for expanders to place code before or after the main form (or do other fancy things):
(defmacro with-decorations ((&rest decorations) form)
(etypecase form
(cons (destructuring-bind (head . tail) form
(ecase head
(defun (destructuring-bind (name args . body) tail
`(progn
,#(loop
for b = `((defun ,name ,args ,#body)) then forms
for d in decorations
for forms = (expand-decoration d name b)
finally (return forms))))))))))
(nb. here above we only care about defun but the loop should probably be done outside of the dispatching thing, along with a way to indicate to expander methods that a function is being expanded; well, it could be better)
Say, for example, you want to declare a function as inline, then the declaration must happen before (so that the compiler can know the source code must be kept):
(defmethod expand-decoration ((_ (eql 'inline)) name rest)
`((declaim (inline ,name)) ,#rest))
Likewise, if you want to export the name of the function being defined, you can export it after the function is defined (order is not really important here):
(defmethod expand-decoration ((_ (eql 'export)) name rest)
`(,#rest (export ',name)))
The resulting code allows you to have a single (progn ...) form with a defun in toplevel position:
CL-USER> (macroexpand '§inline §export (defun my-test-fn () "hello"))
(PROGN
(DECLAIM (INLINE MY-TEST-FN))
(DEFUN MY-TEST-FN () "hello")
(EXPORT 'MY-TEST-FN))
In this post, I ask tangentially why when I declare in SBCL
(defun a (&rest x)
x)
and then check what the function cell holds
(describe 'a)
COMMON-LISP-USER::A
[symbol]
A names a compiled function:
Lambda-list: (&REST X)
Derived type: (FUNCTION * (VALUES LIST &OPTIONAL))
Source form:
(LAMBDA (&REST X) (BLOCK A X))
I see this particular breakdown of the original function. Could someone explain what this output means? I'm especially confused by the last line
Source form:
(LAMBDA (&REST X) (BLOCK A X))
This is mysterious because for some reason not clear to me Lisp has transformed the original function into a lambda expression. It would also be nice to know the details of how a function broken down like this is then called. This example is SBCL. In Elisp
(symbol-function 'a)
gives
(lambda (&rest x) x)
again, bizarre. As I said in the other post, this is easier to understand in Scheme -- but that created confusion in the answers. So once more I ask, Why has Lisp taken a normal function declaration and seemingly stored it as a lambda expression?
I'm still a bit unclear what you are confused about, but here is an attempt to explain it. I will stick to CL (and mostly to ANSI CL), because elisp has a lot of historical oddities which just make things hard to understand (there is an appendix on elisp). Pre-ANSI CL was also a lot less clear on various things.
I'll try to explain things by writing a macro which is a simple version of defun: I'll call this defun/simple, and an example of its use will be
(defun/simple foo (x)
(+ x x))
So what I need to do is to work out what the expansion of this macro should be, so that it does something broadly equivalent (but simpler than) defun.
The function namespace & fdefinition
First of all I assume you are comfortable with the idea that, in CL (and elisp) the namespace of functions is different than the namespace of variable bindings: both languages are lisp-2s. So in a form like (f x), f is looked up in the namespace of function bindings, while x is looked up in the namespace of variable bindings. This means that forms like
(let ((sin 0.0))
(sin sin))
are fine in CL or elisp, while in Scheme they would be an error, as 0.0 is not a function, because Scheme is a lisp-1.
So we need some way of accessing that namespace, and in CL the most general way of doing that is fdefinition: (fdefinition <function name>) gets the function definition of <function name>, where <function name> is something which names a function, which for our purposes will be a symbol.
fdefinition is what CL calls an accessor: this means that the setf macro knows what to do with it, so that we can mutate the function binding of a symbol by (setf (fdefinition ...) ...). (This is not true: what we can access and mutate with fdefinition is the top-level function binding of a symbol, we can't access or mutate lexical function bindings, and CL provides no way to do this, but this does not matter here.)
So this tells us what our macro expansion needs to look like: we want to set the (top-level) definition of the name to some function object. The expansion of the macro should be like this:
(defun/simple foo (x)
x)
should expand to something involving
(setf (fdefinition 'foo) <form which makes a function>)
So we can write this bit of the macro now:
(defmacro defun/simple (name arglist &body forms)
`(progn
(setf (fdefinition ',name)
,(make-function-form name arglist forms))
',name))
This is the complete definition of this macro. It uses progn in its expansion so that the result of expanding it is the name of the function being defined, which is the same as defun: the expansion does all its real work by side-effect.
But defun/simple relies on a helper function, called make-function-form, which I haven't defined yet, so you can't actually use it yet.
Function forms
So now we need to write make-function-form. This function is called at macroexpansion time: it's job is not to make a function: it's to return a bit of source code which will make a function, which I'm calling a 'function form'.
So, what do function forms look like in CL? Well, there's really only one such form in portable CL (this might be wrong, but I think it is true), which is a form constructed using the special operator function. So we're going to need to return some form which looks like (function ...). Well, what can ... be? There are two cases for function.
(function <name>) denotes the function named by <name> in the current lexical environment. So (function car) is the function we call when we say (car x).
(function (lambda ...)) denotes a function specified by (lambda ...): a lambda expression.
The second of these is the only (caveats as above) way we can construct a form which denotes a new function. So make-function-form is going to need to return this second variety of function form.
So we can write an initial version of make-function-form:
(defun make-function-form (name arglist forms)
(declare (ignore name))
`(function (lambda ,arglist ,#forms)))
And this is enough for defun/simple to work:
> (defun/simple plus/2 (a b)
(+ a b))
plus/2
> (plus/2 1 2)
3
But it's not quite right yet: one of the things that functions defined by defun can do is return from themselves: they know their own name and can use return-from to return from it:
> (defun silly (x)
(return-from silly 3)
(explode-the-world x))
silly
> (silly 'yes)
3
defun/simple can't do this, yet. To do this, make-function-form needs to insert a suitable block around the body of the function:
(defun make-function-form (name arglist forms)
`(function (lambda ,arglist
(block ,name
,#forms))))
And now:
> (defun/simple silly (x)
(return-from silly 3)
(explode-the-world x))
silly
> (silly 'yes)
3
And all is well.
This is the final definition of defun/simple and its auxiliary function.
Looking at the expansion of defun/simple
We can do this with macroexpand in the usual way:
> (macroexpand '(defun/simple foo (x) x))
(progn
(setf (fdefinition 'foo)
#'(lambda (x)
(block foo
x)))
'foo)
t
The only thing that's confusing here is that, because (function ...) is common in source code, there's syntactic sugar for it which is #'...: this is the same reason that quote has special syntax.
It's worth looking at the macroexpansion of real defun forms: they usually have a bunch of implementation-specific stuff in them, but you can find the same thing there. Here's an example from LW:
> (macroexpand '(defun foo (x) x))
(compiler-let ((dspec::*location* '(:inside (defun foo) :listener)))
(compiler::top-level-form-name (defun foo)
(dspec:install-defun 'foo
(dspec:location)
#'(lambda (x)
(declare (system::source-level
#<eq Hash Table{0} 42101FCD5B>))
(declare (lambda-name foo))
x))))
t
Well, there's a lot of extra stuff in here, and LW obviously has some trick around this (declare (lambda-name ...)) form which lets return-from work without an explicit block. But you can see that basically the same thing is going on.
Conclusion: how you make functions
In conclusion: a macro like defun, or any other function-defining form, needs to expand to a form which, when evaluated, will construct a function. CL offers exactly one such form: (function (lambda ...)): that's how you make functions in CL. So something like defun necessarily has to expand to something like this. (To be precise: any portable version of defun: implementations are somewhat free to do implementation-magic & may do so. However they are not free to add a new special operator.)
What you are seeing when you call describe is that, after SBCL has compiled your function, it's remembered what the source form was, and the source form was exactly the one you would have got from the defun/simple macro given here.
Notes
lambda as a macro
In ANSI CL, lambda is defined as a macro whose expansion is a suitable (function (lambda ...)) form:
> (macroexpand '(lambda (x) x))
#'(lambda (x) x)
t
> (car (macroexpand '(lambda (x) x)))
function
This means that you don't have to write (function (lambda ...)) yourself: you can rely on the macro definition of lambda doing it for you. Historically, lambda wasn't always a macro in CL: I can't find my copy of CLtL1, but I'm pretty certain it was not defined as one there. I'm reasonably sure that the macro definition of lambda arrived so that it was possible to write ISLisp-compatible programs on top of CL. It has to be in the language because lambda is in the CL package and so users can't portably define macros for it (although quite often they did define such a macro, or at least I did). I have not relied on this macro definition above.
defun/simple does not purport to be a proper clone of defun: its only purpose is to show how such a macro can be written. In particular it doesn't deal with declarations properly, I think: they need to be lifted out of the block & are not.
Elisp
Elisp is much more horrible than CL. In particular, in CL there is a well-defined function type, which is disjoint from lists:
> (typep '(lambda ()) 'function)
nil
> (typep '(lambda ()) 'list)
t
> (typep (function (lambda ())) 'function)
t
> (typep (function (lambda ())) 'list)
nil
(Note in particular that (function (lambda ())) is a function, not a list: function is doing its job of making a function.)
In elisp, however, an interpreted function is just a list whose car is lambda (caveat: if lexical binding is on this is not the case: it's then a list whose car is closure). So in elisp (without lexical binding):
ELISP> (function (lambda (x) x))
(lambda (x)
x)
And
ELISP> (defun foo (x) x)
foo
ELISP> (symbol-function 'foo)
(lambda (x)
x)
The elisp intepreter then just interprets this list, in just the way you could yourself. function in elisp is almost the same thing as quote.
But function isn't quite the same as quote in elisp: the byte-compiler knows that, when it comes across a form like (function (lambda ...)) that this is a function form, and it should byte-compile the body. So, we can look at the expansion of defun in elisp:
ELISP> (macroexpand '(defun foo (x) x))
(defalias 'foo
#'(lambda (x)
x))
(It turns out that defalias is the primitive thing now.)
But if I put this definition in a file, which I byte compile and load, then:
ELISP> (symbol-function 'foo)
#[(x)
"\207"
[x]
1]
And you can explore this a bit further: if you put this in a file:
(fset 'foo '(lambda (x) x))
and then byte compile and load that, then
ELISP> (symbol-function 'foo)
(lambda (x)
x)
So the byte compiler didn't do anything with foo because it didn't get the hint that it should. But foo is still a fine function:
ELISP> (foo 1)
1 (#o1, #x1, ?\C-a)
It just isn't compiled. This is also why, if writing elisp code with anonymous functions in it, you should use function (or equivalently #'). (And finally, of course, (function ...) does the right thing if lexical scoping is on.)
Other ways of making functions in CL
Finally, I've said above that function & specifically (function (lambda ...)) is the only primitive way to make new functions in CL. I'm not completely sure that's true, especially given CLOS (almost any CLOS will have some kind of class instances of which are functions but which can be subclassed). But it does not matter: it is a way and that's sufficient.
DEFUN is a defining macro. Macros transform code.
In Common Lisp:
(defun foo (a)
(+ a 42))
Above is a definition form, but it will be transformed by DEFUN into some other code.
The effect is similar to
(setf (symbol-function 'foo)
(lambda (a)
(block foo
(+ a 42))))
Above sets the function cell of the symbol FOO to a function. The BLOCK construct is added by SBCL, since in Common Lisp named functions defined by DEFUN create a BLOCK with the same name as the function name. This block name can then be used by RETURN-FROM to enable a non-local return from a specific function.
Additionally DEFUN does implementation specific things. Implementations also record development information: the source code, the location of the definition, etc.
Scheme has DEFINE:
(define (foo a)
(+ a 10))
This will set FOO to a function object.
I am trying to make a list of callback functions, which could look like this:
(("command1" . 'callback1)
("command2" . 'callback2)
etc)
I'd like it if I could could do something like:
(define-callback callback1 "command1" args
(whatever the function does))
Rather than
(defun callback1 (args)
(whatever the function does))
(add-to-list 'callback-info ("command1" . 'callback1))
Is there a convenient way of doing this, e.g., with macros?
This is a good example of a place where it's nice to use a two-layered approach, with an explicit function-based layer, and then a prettier macro layer on top of that.
Note the following assumes Common Lisp: it looks just possible from your question that you are asking about elisp, in which case something like this can be made to work but it's all much more painful.
First of all, we'll keep callbacks in an alist called *callbacks*:
(defvar *callbacks* '())
Here's a function which clears the alist of callbacks
(defun initialize-callbacks ()
(setf *callbacks* '())
(values)
Here is the function that installs a callback. It does this by searching the list to see if there is a callback with the given name, and if there is then replacing it, and otherwise installing a new one. Like all the functions in the functional layer lets us specify the test function which will let us know if two callback names are the same: by default this is #'eql which will work for symbols and numbers, but not for strings. Symbols are probably a better choice for the names of callbacks than strings, but we'll cope with that below.
(defun install-callback (name function &key (test #'eql))
(let ((found (assoc name *callbacks* :test test)))
(if found
(setf (cdr found) function)
(push (cons name function) *callbacks*)))
name)
Here is a function to find a callback, returning the function object, or nil if there is no callback with that name.
(defun find-callback (name &key (test #'eql))
(cdr (assoc name *callbacks* :test test)))
And a function to remove a named callback. This doesn't tell you if it did anything: perhaps it should.
(defun remove-callback (name &key (test #'eql))
(setf *callbacks* (delete name *callbacks* :key #'car :test test))
name)
Now comes the macro layer. The syntax of this is going to be (define-callback name arguments ...), so it looks a bit like a function definition.
There are three things to know about this macro.
It is a bit clever: because you can know at macro-expansion time what sort of thing the name of the callback is, you can decide then and there what test to use when installing the callback, and it does this. If the name is a symbol it also wraps a block named by the symbol around the body of the function definition, so it smells a bit more like a function defined by defun: in particular you can use return-from in the body. It does not do this if the name is not a symbol.
It is not quite clever enough: in particular it does not deal with docstrings in any useful way (it ought to pull them out of the block I think). I am not sure this matters.
The switch to decide the test uses expressions like '#'eql which reads as (quote (function eql)): that is to avoid wiring in functions into the expansion because functions are not externalisable objects in CL. However I am not sure I have got this right: I think what is there is safe but it may not be needed.
So, here it is
(defmacro define-callback (name arguments &body body)
`(install-callback ',name
,(if (symbolp name)
`(lambda ,arguments
(block ,name
,#body))
`(lambda ,arguments
,#body))
:test ,(typecase name
(string '#'string=)
(symbol '#'eql)
(number '#'=)
(t '#'equal))))
And finally here are two different callbacks being defined:
(define-callback "foo" (x)
(+ x 3))
(define-callback foo (x)
(return-from foo (+ x 1)))
These lists are called assoc lists in Lisp.
CL-USER 120 > (defvar *foo* '(("c1" . c1) ("c2" . c2)))
*FOO*
CL-USER 121 > (setf *foo* (acons "c0" `c1 *foo*))
(("c0" . C1) ("c1" . C1) ("c2" . C2))
CL-USER 122 > (assoc "c1" *foo* :test #'equal)
("c1" . C1)
You can write macros for that, but why? Macros are advanced Lisp and you might want to get the basics right, first.
Some issues with you example you might want to check out:
what are assoc lists?
what are useful key types in assoc lists?
why you don't need to quote symbols in data lists
variables are not quoted
data lists need to be quoted
You can just as easy create such lists for callbacks without macros. We can imagine a function create-callback, which would be used like this:
(create-callback 'callback1 "command1"
(lambda (arg)
(whatever the function does)))
Now, why would you use a macro instead of a plain function?
In the end, assisted by the responders above, I got it down to something like:
(defmacro mk-make-command (name &rest body)
(let ((func-sym (intern (format "mk-cmd-%s" name))))
(mk-register-command name func-sym)
`(defun ,func-sym (args &rest rest)
(progn
,#body))))
I am trying to emulate the single namespace of scheme within common lisp, with a macro (based on Doug Hoyte's) that expands to a lambda, where every use of an f! symbol (similar to Doug Hoyte's o! and g! symbols) in the function position expands to the same expression, but with funcall added in the function position of each invocation. For example:
(fplambda (f!z x) (f!z x x))
would expand to:
(LAMBDA (F!Z X) (FUNCALL F!Z X X))
The macro currently looks like this:
(defmacro fplambda (parms &body body)
(let ((syms (remove-duplicates
(remove-if-not #'f!-symbol-p
(flatten body)))))
`(lambda ,parms
(macrolet ,(mapcar
(lambda (f)
`(,f (&rest parmlist) `(funcall ,',f ',#parmlist)))
syms))
,#body)))
but given the above input, it expands (as far as I can see) to this:
(LAMBDA (F!F X)
(MACROLET ((F!F (&REST PARMLIST) `(FUNCALL ,'F!F ',#PARMLIST))))
(F!F X X))
In the macrolet definition, F!F should not be quoted or unquoted, and parmlist should just be unquoted. What is going on?
Thanks in advance!
Your definition is mostly right. You just made two pretty simple mistakes. The first one being a mismatched paren. The macrolet does not include the body (in the output the macrolet and the body are at the same level of indentation).
As for the nested backquote, the only mistake is the quote before parmlist. Other than that everything else is correct. The comma and quote before F!F is actually correct. From the hyperspec:
"An implementation is free to interpret a backquoted form F1 as any form F2 that, when evaluated, will produce a result that is the same under equal as the result implied by the above definition". Since the inner backquote has not been expanded yet, it does not have to be free of quotes and unquotes. The expression `(,'x) is actually the same as `(x).
Nested backquotes are notoriously complicated. What is probably the easiest way to understand them is to read Steele's explanation of them.
Edit:
The answer to your question about whether it is possible to use a fplambda expression in the function position is no. From the part of the hyperspec that deals with the evaluation of code: "If the car of the compound form is not a symbol, then that car must be a lambda expression, in which case the compound form is a lambda form.". Since the car of the form, (fplambda ...), is not a lambda expression, your code is no longer valid Common Lisp code.
There is a workaround to this that I figured out, but it's kind of ugly. You can define a reader macro that will allow you to write something like ([fplambda ...] ...) and have it read as
((LAMBDA (&REST #:G1030) (APPLY (FPLAMBDA ...) #:G1030)) ...)
which would do what you want. Here is code that will allow you to do that:
(set-macro-character #\[ 'bracket-reader)
(set-macro-character #\] (get-macro-character #\)))
(defun bracket-reader (stream char)
"Read in a bracket."
(declare (ignore char))
(let ((gargs (gensym)))
`(lambda (&rest ,gargs)
(apply ,(read-delimited-list #\] stream t)
,gargs))))
The only other solution I can think of would be to use some sort of code walker (I can't help you there).
Lisps often declare, that certain types are self-evaluating. E.g. in emacs-lisp numbers, "strings", :keyword-symbols and some more evaluate to themselves.
Or, more specifically: Evaluating the form and evaluating the result again gives the same result.
It is also possible to create custom self-evaluating forms, e.g.
(defun my-list (&rest args)
(cons 'my-list (mapcar (lambda (e) (list 'quote e)) args)))
(my-list (+ 1 1) 'hello)
=> (my-list '2 'hello)
(eval (my-list (+ 1 1) 'hello))
=> (my-list '2 'hello)
Are there any practical uses for defining such forms or is this more of an esoteric concept?
I thought of creating "custom-types" as self-evaluating forms, where the evaluation may for instance perform type-checks on the arguments. When trying to use such types in my code, I usually found it inconvenient compared to simply working e.g. with plists though.
*edit* I checked again, and it seems I mixed up "self-evaluating" and "self-quoting". In emacs lisp the later term was applied to the lambda form, at least in contexts without lexical binding. Note that the lambda form does never evaluate to itself (eq), even if the result is equal.
(setq form '(lambda () 1)) ;; => (lambda () 1)
(equal form (eval form)) ;; => t
(equal (eval form) (eval (eval form))) ;; => t
(eq form (eval form)) ;; => nil
(eq (eval form) (eval (eval form))) ;; => nil
As Joshua put it in his answer: Fixed-points of the eval function (with respect to equal).
The code you presented doesn't define a type of self-evaluating form. A self evaluating form that eval would return when passed as an argument. Let's take a closer look. First, there's a function that takes some arguments and returns a new list:
(defun my-list (&rest args)
(cons 'my-list (mapcar (lambda (e) (list 'quote e)) args)))
The new list has the symbol my-list as the first elements. The remaining elements are two-element lists containing the symbol quote and the elements passed to the function:
(my-list (+ 1 1) 'hello)
;=> (my-list '2 'hello)
Now, this does give you a fixed point for eval with regard to equal, since
(eval (my-list (+ 1 1) 'hello))
;=> (my-list '2 'hello)
and
(eval (eval (my-list (+ 1 1) 'hello)))
;=> (my-list '2 'hello)
It's also the case that self-evaluating forms are fixed points with respect to equals, but in Common Lisp, a self-evaluating form is one that is a fixed point for eval with respect to eq (or perhaps eql).
The point of the language specifying self-evaluating forms is really to define what the evaluator has to do with forms. Conceptually eval would be defined something like this:
(defun self-evaluating-p (form)
(or (numberp form)
(stringp form)
(and (listp form)
(eql 2 (length form))
(eq 'quote (first form)))
; ...
))
(defun eval (form)
(cond
((self-evaluating-p form) form)
((symbolp form) (symbol-value-in-environment form))
;...
))
The point is not that a self-evaluating form is one that evaluates to an equivalent (for some equivalence relation) value, but rather one for which eval doesn't have to do any work.
Compiler Macros
While there's generally not a whole lot of use for forms that evaluate to themselves (modulo some equivalence) relation, there is one very important place where something very similar is used Common Lisp: compiler macros (emphasis added):
3.2.2.1 Compiler Macros
The function returned by compiler-macro-function is a function of two
arguments, called the expansion function. To expand a compiler macro,
the expansion function is invoked by calling the macroexpand hook with
the expansion function as its first argument, the entire compiler
macro form as its second argument, and the current compilation
environment (or with the current lexical environment, if the form is
being processed by something other than compile-file) as its third
argument. The macroexpand hook, in turn, calls the expansion function
with the form as its first argument and the environment as its second
argument. The return value from the expansion function, which is
passed through by the macroexpand hook, might either be the same form,
or else a form that can, at the discretion of the code doing the
expansion, be used in place of the original form.
Macro DEFINE-COMPILER-MACRO
Unlike an ordinary macro, a compiler macro can decline to provide an expansion merely by returning a form that is the same as the original
(which can be obtained by using &whole).
As an example:
(defun exponent (base power)
"Just like CL:EXPT, but with a longer name."
(expt base power))
(define-compiler-macro exponent (&whole form base power)
"A compiler macro that replaces `(exponent base 2)` forms
with a simple multiplication. Other invocations are left the same."
(if (eql power 2)
(let ((b (gensym (string '#:base-))))
`(let ((,b ,base))
(* ,b ,b)))
form))
Note that this isn't quite the same as a self-evaluating form, because the compiler is still going through the process of checking whether a form is a cons whose car has an associated compiler macro, and then calling that compiler macro function with the form. But it's similar in that the form goes to something and the case where the same form comes back is important.
What you describe and self-evaluating forms (not types!) is unrelated.
? (list (foo (+ 1 2)))
may evaluate to
-> (foo 3)
But that's running the function foo and it is returning some list with the symbol foo and its first argument value. Nothing more. You've written a function. But not a custom self evaluating form.
A form is some data meant to be evaluated. It needs to be valid Lisp code.
About Evaluation of Forms:
Evaluation of forms is a topic when you have source like this:
(defun foo ()
(list #(1 2 3)))
What's with the above vector? Does (foo) return a list with the vector as its first element?
In Common Lisp such vector forms are self-evaluating. In some other Lisps it was different. In some older Lisp dialect one probably had to write the code below to make the compiler happy. It might even be different with an interpreter. (I've seen this loooong ago in some implementation of a variant of Standard Lisp).
(defun foo ()
(list '#(1 2 3))) ; a vector form quoted
Note the quote. Non-self evaluating forms had to be quoted. That's relatively easy to do. You have to look at the source code and make sure that such forms are quoted. But there is another problem which makes it more difficult. Such data objects could have been introduced by macros in the code. Thus one also had to make sure that all code generated by macros has all literal data quoted. Which makes it a real pain.
This was wrong in some other Lisp dialect (not in Common Lisp):
(defmacro foo (a)
(list 'list a #(1 2 3)))
or even (note the added quote)
(defmacro foo (a)
(list 'list a '#(1 2 3)))
Using
(foo 1)
would be the code (list 1 #(1 2 3)). But in these Lisps there would be a quote missing... so it was wrong there.
One had to write:
(defmacro foo (a)
(list 'list a ''#(1 2 3))) ; note the double quote
Thus
(foo 1)
would be the code (list 1 '#(1 2 3)). Which then works.
To get rid of such problems, Lisp dialects like Common Lisp required that all forms other than symbols and conses are self evaluating. See the CL standard: Self-Evaluating Objects. This is also independent of using an interpreter or compiler.
Note that Common Lisp also provides no mechanism to change that.
What could be done with a custom mechanim? One could let data forms evaluate to something different. Or one could implement different evaluation schemes. But there is nothing like that in Common Lisp. Basically we've got symbols as variables, conses as special forms / functions / macros and the rest is self-evaluating. For anything different you would need to write a custom evaluator/compiler.