Edit: In my examples, => means "evaluates to", and -> means "expands to under macroexpand-1".
I'm trying to wrap my head around nested back-quoting in Common Lisp, and I think I'm very close to understanding it, thanks to several other SO questions (please don't refer me to them--I've seen them). There's just one last thing that's nagging me. Consider:
`(a `(b ,,#(list 'c 'd)))
=> (a `(b ,c ,d))
Following the algorithm for expanding back-quoted forms in CLHS, I stepped through the (read-time) expansion process of the first form above, and indeed, evaluating the form obtained from that expansion does give an equivalent result. Now, consider the definition of once-only given by Peter Seibel in his excellent book.
(defmacro once-only ((&rest names) &body body)
(let ((gensyms (loop for n in names collect (gensym))))
`(let (,#(loop for g in gensyms collect `(,g (gensym))))
`(let (,,#(loop for g in gensyms for n in names collect ``(,,g ,,n)))
,(let (,#(loop for n in names for g in gensyms collect `(,n ,g)))
,#body)))))
Given that definition, SBCL shows the following macro expansion:
(once-only (from to)
`(do ((,var ,from (next-prime (+ 1 ,var))))
((>= ,var ,to))
,#body)))
-> (LET ((#:G939 (GENSYM)) (#:G940 (GENSYM)))
`(LET ((,#:G939 ,FROM) (,#:G940 ,TO))
,(LET ((FROM #:G939) (TO #:G940))
`(DO ((,VAR ,FROM (NEXT-PRIME (+ ,1 ,VAR)))) ((>= ,VAR ,TO)) ,#BODY))))
Now, my issue is with the second line of the expanded form. Shouldn't it be:
`(LET (,(,#:G939 ,FROM) ,(,#:G940 ,TO))
?
See how the evaluation of the ,,# in the first example above ends up "distributing" the comma to the elements spliced from the list (c d)? Why doesn't that occur here? The two examples seem to share the same structure, but their evaluated results seem inconsistent.
I think I see my error. The second line in the macro-expansion above should actually be:
`(LET (,`(,#:G939 ,FROM) ,`(,#:G940 ,TO))
which is equivalent to what SBCL shows.
Related
I'm learning common lisp. I have written a version of the once-only macro, which suffers from an unusual variable capture problem.
My macro is this:
(defmacro my-once-only (names &body body)
(let ((syms (mapcar #'(lambda (x) (gensym))
names)))
``(let (,,#(mapcar #'(lambda (sym name) ``(,',sym ,,name))
syms names))
,(let (,#(mapcar #'(lambda (name sym) `(,name ',sym))
names syms))
,#body))))
The canonical version of only-once is this:
(defmacro once-only ((&rest names) &body body)
(let ((gensyms (loop for n in names collect (gensym))))
`(let (,#(loop for g in gensyms collect `(,g (gensym))))
`(let (,,#(loop for g in gensyms for n in names collect ``(,,g ,,n)))
,(let (,#(loop for n in names for g in gensyms collect `(,n ,g)))
,#body)))))
The difference, as far as I can tell, is that the canonical version generates new symbols for every expansion of the macro using only-once. For example:
CL-USER> (macroexpand-1 '(once-only (foo) foo))
(LET ((#:G824 (GENSYM)))
`(LET (,`(,#:G824 ,FOO))
,(LET ((FOO #:G824))
FOO)))
T
CL-USER> (macroexpand-1 '(my-once-only (foo) foo))
`(LET (,`(,'#:G825 ,FOO))
,(LET ((FOO '#:G825))
FOO))
T
The variable my macro uses to store the value of foo is the same for every expansion of this form, in this case it would be #:G825. This is akin to defining a macro like the following:
(defmacro identity-except-for-bar (foo)
`(let ((bar 2))
,foo))
This macro captures bar, and this capture manifests when bar is passed to it, like so:
CL-USER> (let ((bar 1))
(identity-except-for-bar bar))
2
However, I cannot think of any way to pass #:G825 to a macro that uses my-only-once so that it breaks like this, because the symbols gensym returns are unique, and I cannot create a second copy of it outside of the macro. I assume that capturing it is unwanted, otherwise the canonical version wouldn't bother adding the additional layer of gensym. How could capturing a symbol like #:G826 be a problem? Please provide an example where this capture manifests.
We can demonstrate a behavioral difference between my-once-only and once-only:
Let's store our test form in a variable.
(defvar *form* '(lexalias a 0 (lexalias b (1+ a) (list a b))))
This test form exercises a macro called lexalias, which we will define in two ways. First with once-only:
(defmacro lexalias (var value &body body)
(once-only (value)
`(symbol-macrolet ((,var ,value))
,#body)))
(eval *form*) -> (0 1)
Then with my-once-only:
(defmacro lexalias (var value &body body)
(my-once-only (value)
`(symbol-macrolet ((,var ,value))
,#body)))
(eval *form*) -> (1 1)
Oops! The problem is that under my-once-only, both a and b end up being symbol-macrolet aliases for exactly the same gensym; the returned expression (list a b) ends up being something like (list #:g0025 #:g0025).
If you're writing a macro-writing helper that implements once-only evaluation, you have no idea how the symbol is going to be used by the code which calls the macro, whose author uses your once-only tool. There are two big unknowns: the nature of the macro and of its use.
As you can see, if you don't make fresh gensyms, it will not work correctly in all conceivable scenarios.
I am trying to expand all macros inside a nested list structure.
macroexpand-all almost works, but skips (does not expand) the first form in every nested list.
I am using this as a template-mechanism for org-agenda-custom-commands. I can generate agenda-blocks for multiple agenda-commands via macros. This is in init.el (emacs26.2).
macroexp-all-forms is able to not skip the first form, but calls macroexpand-all for nested forms.
Here a minimal example from the emacs doc:
(defmacro inc (var)
(list 'setq var (list '1+ var)))
This works as expected (one macro-call):
ELISP> (macroexpand-all '(inc r))
(setq r
(1+ r))
This works too (nested, but first form is not a macro-call):
ELISP> (macroexpand-all '(('foo)(inc r)))
(('foo)
(setq r
(1+ r)))
This does NOT work (nested and first form is a macro-call):
ELISP> (macroexpand-all '((inc r)(inc r)))
((inc r)
(setq r
(1+ r)))
This also does not work:
ELISP> (macroexpand-all '((inc r)))
((inc r))
In the last two examples, the first call to inc is not expanded. What am I missing here?
How can i really expand all macros in this situation?
macroexpand-all expects a form as the argument, but ((inc r) (inc r)) is not a form. A form should be something that can be evaluated. In the case of a list, that means the first element must be a name of a function, macro or a special operator, or a lambda expression.
The special operator progn can be used for a list of forms to be evaluated sequentially. For example:
(macroexpand-all '(progn (inc r) (inc r)))
;=> (progn
; (setq r
; (1+ r))
; (setq r
; (1+ r)))
Alternatively, if the list is not meant to be a form, you can use mapcar to apply macroexpand-all to each member of the list. For example:
(mapcar #'macroexpand-all '((inc r) (inc r)))
;=> ((setq r
; (1+ r))
; (setq r
; (1+ r)))
Keep in mind that the result here is a list of forms, but not a form itself; it cannot be evaluated as is, but the individual elements can be.
The form you want to expand should make sense as a form for evaluation. ((inc r)) does not: it can't be legal elisp.
I'm actually fairly surprised that macroexpand-all doesn't raise an error in the case you've given it. If you want it to work you need to fake up something which is potentially legal. For instance
ELISP> (macroexpand-all '(grut (inc r) (inc r)))
(grut
(setq r
(1+ r))
(setq r
(1+ r)))
ELISP> (cdr (macroexpand-all '(grut (inc r) (inc r))))
((setq r
(1+ r))
(setq r
(1+ r)))
grut does not need to be defined: it only needs not to be a macro so macroexpand-all just ignores it. (In fact, using progn as jkiiski suggests is a better answer).
As an aside, it sounds like you are using macroexpansion to do something other than expand Lisp code: that's almost always a bad idea. It's much better to write your own expander which understands the rules of the language you are implementing in Lisp.
At the end of Ch 8 in Practical Common Lisp, Peter Seibel presents the once-only macro. Its purpose is to mitigate a number of subtle problems with variable evaluation in user-defined macros. Note I'm not trying to understand at this point how this macro works, as in some other posts, but just how to use it properly:
(defmacro once-only ((&rest names) &body body)
(let ((gensyms (loop for n in names collect (gensym))))
`(let (,#(loop for g in gensyms collect `(,g (gensym))))
`(let (,,#(loop for g in gensyms for n in names collect ``(,,g ,,n)))
,(let (,#(loop for n in names for g in gensyms collect `(,n ,g)))
,#body)))))
The following is a sample (incorrect) contrived macro that attempts to exhibit several variable evaluation problems. It purports to iterate over a range of integers by some delta, returning the range:
(defmacro do-range ((var start stop delta) &body body)
"Sample macro with faulty variable evaluations."
`(do ((,var ,start (+ ,var ,delta))
(limit ,stop))
((> ,var limit) (- ,stop ,start))
,#body))
For example, (do-range (i 1 15 3) (format t "~A " i)) should print 1 4 7 10 13 and then return 14.
The problems include 1) potential capture of the second occurrence of limit, since it occurs as a free variable, 2) potential capture of the initial occurrence of the bound variable limit, since it occurs in an expression along with other variables appearing in the macro parameters, 3) out of order evaluation, since delta will be evaluated before stop, even though stop appears before delta in the parameter list, and 4) multiple variable evaluations, since stop and start are evaluated more than once. As I understand it, once-only should fix these problems:
(defmacro do-range ((var start stop delta) &body body)
(once-only (start stop delta limit)
`(do ((,var ,start (+ ,var ,delta))
(limit ,stop))
((> ,var limit) (- ,stop ,start))
,#body)))
However, (macroexpand '(do-range (i 1 15 3) (format t "~A " i))) complains about limit being an unbound variable. If I switch instead to with-gensyms, which should take care of problems 1 & 2 above only, the expansion proceeds without incident.
Is this an issue with the once-only macro? And does once-only really solve all the problems outlined above (and perhaps others)?
The ONCE-ONLY macro
To get rid of a warning that N is unused, I would change the macro to:
(defmacro once-only ((&rest names) &body body)
(let ((gensyms (loop for nil in names collect (gensym))))
; changed N to NIL, NIL is ignored
`(let (,#(loop for g in gensyms collect `(,g (gensym))))
`(let (,,#(loop for g in gensyms for n in names collect ``(,,g ,,n)))
,(let (,#(loop for n in names for g in gensyms collect `(,n ,g)))
,#body)))))
The purpose of this macro is to make sure that expressions are only evaluated once and in a defined order. For that it will introduce new uninterned variables and will bind the evaluation results to those. Inside he macro the new variables are available. The macro itself is provided to make writing macros easier.
Using ONCE-ONLY in DO-RANGE
Your example use of ONCE-ONLY:
(defmacro do-range ((var start stop delta) &body body)
(once-only (start stop delta limit)
`(do ((,var ,start (+ ,var ,delta))
(limit ,stop))
((> ,var limit) (- ,stop ,start))
,#body)))
Why is there LIMIT in the once-only list? limit is undefined there. LIMIT is used inside the ONCE-ONLY form as a symbol, but outside there is no binding.
ONCE-ONLY expects that the list of names is a list of symbols and that these names are bound to forms. In your case limit is a symbol, but it is undefined.
We need to remove limit from the list of names:
(defmacro do-range ((var start stop delta) &body body)
(once-only (start stop delta)
`(do ((,var ,start (+ ,var ,delta))
(limit ,stop))
((> ,var limit) (- ,stop ,start))
,#body)))
Now, what to do about LIMIT? Given that once-only provides bindings for the names, including for STOP, we can eliminate the symbol LIMIT and replace its use with ,stop:
(defmacro do-range ((var start stop delta) &body body)
(once-only (start stop delta)
`(do ((,var ,start (+ ,var ,delta)))
((> ,var ,stop) (- ,stop ,start))
,#body)))
Example:
CL-USER 137 > (pprint
(macroexpand
'(do-range (i 4 10 2)
(print i))))
(LET ((#1=#:G2170 4)
(#3=#:G2171 10)
(#2=#:G2172 2))
(DO ((I #1# (+ I #2#)))
((> I #3#) (- #3# #1#))
(PRINT I)))
Looking at Practical Common Lisp, we're looking at a simple automated unit test framework. We're trying to write a macro to be used as such:
(check (= (+ 1 2) 3) (= (- 1 4) 9))
This should expand to something using a previously defined function report-result. The suggested implementation is:
(defmacro check (&body forms)
`(progn
,#(loop for f in forms collect `(report-result ,f ',f))))
However, that expansion seems rather procedural to me. I wanted to replace the loop with a mapcar to expand to something like this:
(mapcar #'(lambda (form) (report-result form 'form)) (list form-1 ... form-n))
However, I'm clearly lacking the macro-writing skills to do so. Can someone come up with one such macro?
In case it's relevant, this is the definition of report-result:
(defun report-result (result form)
(format t "~:[FAIL~;pass~] ... ~a~%" result form))
It's indeed fairly simple: you just place the collect expression into the body of your mapcar:
(defmacro check (&body forms)
`(progn
,#(mapcar #'(lambda (form)
`(report-result ,form ',form))
forms)))
You don't really need to know anything about the "macro-y" stuff that's going on, in order to do the replacement you want, which is simply replacing a loop with some other equivalent expression: it will work just as well in a macro context as it would outside.
If you want to expand to a mapcar you can, but there's no real reason to do so, since the list's size is known at compile time. Here's what that would look like:
(defmacro check (&body forms)
`(let ((results (list ,#(mapcar #'(lambda (form)
`(list ,form ',form))
forms))))
(mapcar #'(lambda (result)
(report-result (car result) (cadr result)))
results)))
Which expands like so
> (macroexpand-1 '(check (+ 1 2) (* 2 3)))
(let ((results (list (list (+ 1 2) '(+ 1 2))
(list (* 2 3) '(* 2 3)))))
(mapcar #'(lambda (result) (report-result (car result) (cadr result)))
results))
Which as you can see is rather awkward: the macro already has the forms like (+ 1 2) available to it, but in order to preserve them to runtime for the mapcar lambda to see, you have to emit the input form twice. And you have to produce the whole list to map over, rather than just producing a list that's "finished" to begin with. Additionally, this produces a list as output, and requires having all the inputs and outputs in memory at once: the original macro with progn produced the inputs and outputs one at a time, and discarded them when finished.
I am quite new to Common Lisp and programming, and I'm trying to write a certain function that turns all non-nil args into an alist. The only way I can think of so far is:
(let ((temp nil))
(if arg1
(setf temp (acons 'arg1 arg1 nil)))
(if arg2
(setf temp (acons 'arg2 arg2 temp)))
...
(if arg20-ish
(setf temp (acons 'arg20-ish arg20-ish temp)))
(do-something-with temp))
which does not seem very elegant, it would be messy with many arguments and when these need to be changed. I am looking for a smarter way to do this, both for the sake of writing this particular function and for learning how to think in Lisp and/or functional programming.
The tricky part for me is figuring out how to get the names of the arguments or what symbol to use, without hand coding each case. If &rest provided arg names it would be easy to filter out NILs with loop or mapcar, but since it doesn't, I can't see how to "automate" this.
I'm totally interested in other solutions than the one described, if people think this way is unnatural.
Edit: Below is an example of what I am trying to do:
An object is created, with a non-fixed number of data pairs and some tags, e.g.:
user = "someone"
creation-time = (get-universal-time)
color-of-sky = "blue"
temperature-in-celsius = 32
language = "Common Lisp"
...
tags = '("one" "two" "three")
These properties (i.e. key/arg names) could be different each time. The new object will then be added to a collection; I thought the array might work well since I want constant access time and only need a numeric ID.
The collection will hold more and more such custom objects, indefinitely.
I want to be able to quickly access all objects matching any combination of any of the tags used in these objects.
Since the array is supposed to store more and more data over a long period, I don't want to parse every item in it each time I need to search for a tag. Thus I also store the index of each object with a given tag in a hash-table, under the tag name. I have written this function, what I find difficult is figuring out how to collect the data and turn it into an alist or anything that I can easily parse, index, and store.
This macro will define a function that turns its non-nil arguments into an alist bound during execution of the body:
(defmacro defnamed (fun-name alist-sym (&rest args) &body body)
`(defun ,fun-name (,#args)
(let ((,alist-sym))
,#(mapcar
(lambda (s)
`(when ,s
(push (cons ',s ,s) ,alist-sym)))
(reverse args))
,#body)))
Demonstration:
(defnamed make-my alist (a b c)
alist)
(make-my 1 NIL 3)
=> ((A . 1) (C . 3))
Here's a sort of solution using macros:
(defmacro named-args (fun-name alist-sym (&rest syms) &body body)
`(defun ,fun-name (&key ,#syms)
(declare (special ,#syms))
(let ((,alist-sym
(loop
for s in ',syms
collecting (cons s (symbol-value s)))))
,#body)))
You can then use it with something like
(named-args f u (a b c)
(format t "~A~%" u))
which expands to
(DEFUN F (&KEY A B C)
(DECLARE (SPECIAL A B C))
(LET ((U
(LOOP FOR S IN '(A B C)
COLLECTING (CONS S (SYMBOL-VALUE S)))))
(FORMAT T "~A~%" U)))
Finally, calling will give
(f :a 3) => ((A . 3) (B) (C))
Note that we need the special declaration otherwise symbol-value doesn't work (you need a global binding for symbol-value). I couldn't find a way to get rid of that.
Looking at your question again, it looks like you actually don't want the keyword arguments that didn't get passed. In which case you could parse a &rest argument (although that's a flat list, so you'd need to map along it in twos) or you could modify the macro as follows:
(defmacro named-args (fun-name alist-sym (&rest syms) &body body)
`(defun ,fun-name (&key ,#syms)
(declare (special ,#syms))
(let ((,alist-sym
(loop
for s in ',syms
when (symbol-value s)
collecting (cons s (symbol-value s)))))
,#body)))
and then you get
(f :a 3) => ((A . 3))