Is there a way to get a macro to do an extra evaluation before returning its result? - macros

I’m trying to get get my macro to do an extra evaluation of its result before returning it. Can this be done without eval?
I'm trying to solve the problem in exercise 4 below:
Define a macro nth-expr that takes an integer n and an arbitrary number of expressions, evaluates the nth expression and returns its value. This exercise is easy to solve, if you assume that the first argument is a literal integer.
4. As exercise 3, but assume that the first argument is an expression to be evaluated.
It's easy to get the macro to pick the right expression:
(defmacro nth-expr% (n &rest es)
`(nth ,n ',es))
CL-USER> (defvar i 1)
I
CL-USER> (nth-expr% (1+ i) (+ 2 3) (- 4 3) (+ 3 1))
(+ 3 1)
The expression (+ 3 1) is the one we want, but we want the macro to evaluate it to 4 before returning it.
It can of course be done with eval:
(defmacro nth-expr%% (n &rest es)
`(eval (nth ,n ',es)))
CL-USER> (nth-expr%% (1+ i) (+ 2 3) (- 4 3) (+ 3 1))
4
But is there another way?
It feels like the solution should be to put the body of nth-expr% in a helper macro and have the top level macro only contain an unquoted call to this helper:
(defmacro helper (n es)
`(nth ,n ',es))
(defmacro nth-expr (n &rest es) ; doesn't work!
(helper n es))
The idea is that the call to helper would return (+ 3 1), and this would then be the expansion of the call to nth-expr, which at run-time would evaluate to 4. It blows up, of course, because N and ES get treated like literals.

That's not that easy.
Using eval is not good, since eval does not evaluate the code in the local lexical environment.
Remember, if we allow an expression to be evaluated to determine the number of another expression to execute, then we don't know this number at macro expansion time - since the expression could be based on a value that needs to be computed - for example based on some variable:
(nth-expression
foo
(bar)
(baz))
So we might want to think about code which does that:
(case foo
(0 (bar))
(1 (baz)))
CASE is evaluating foo and then uses the result to find a clause which has the same value in its head. The consequent forms of that clause then will be evaluated.
Now we need to write code which expands the former into the latter.
This would be a very simple version:
(defmacro nth-expression (n-form &body expressions)
`(case ,n-form
,#(loop for e in expressions
and i from 0
collect `(,i ,e))))
Question: what might be drawbacks of using CASE like that?

Knuto: Rainer Joswig may be asking you to think about how the case statement works. Namely, that after evaluating the keyform (ie, the first argument), it will be compared sequentially to the key in each clause until a match is found. The comparisons could be time consuming if there are many clauses. You can discover this by carefully reading the entry for case in the Hyperspec (as he more than once has insisted I do):
The keyform or keyplace is evaluated to produce the test-key. Each of
the normal-clauses is then considered in turn.
Also note that constructing many case clauses will add to the time to expand and compile the macro at compile time.
Regarding your use of eval in nth-expr%%, you can still achieve the effect of an eval by switching to apply:
(defmacro nth-expr%% (n &rest es)
`(let ((ne (nth ,n ',es)))
(apply (car ne) (cdr ne))))
But see Plugging the Leaks at http://www.gigamonkeys.com/book/macros-defining-your-own.html about a more robust treatment.
In general, a more efficient way to process the expressions is as a simple vector, rather than a list. (The problem statement does not rule out a vector representation.) While nth and case involve searching through the expressions one-by-one, a function like aref or svref can directly index into it. Assuming a vector of expressions is passed to the macro along with an index, perhaps first requiring (coerce expressions 'simple-vector) if a list, then the result can be computed in constant time no matter how many expressions there are:
(defmacro nth-expr%%% (n es)
`(let ((ne (svref ',es ,n)))
(apply (car ne) (cdr ne))))
so that now
(defvar i 1)
(nth-expr%%% (1+ i) #((+ 2 3) (- 4 3) (+ 3 1))) -> 4

Related

LISP Sum of list items

I wrote this macro that rewrites e.g. (sum-expr (1 2 3)) as (+ 1 2 3):
(defmacro sum-expr (expr-list)
`(+ ,#expr-list))
=> SUM-EXPR
For example:
(sum-expr ((+ 1 3) (* 3 4) (- 8 4)))
=> 20
How can I define an equivalent function using defun?
As #LeonardoDagnino already mentioned it in comment:
(defun sum-expr (lst)
(apply #'+ lst))
Would be very lispy, but implementation-dependent CALL-ARGUMENTS-LIMIT limits in some implementations the length of lst to 50 as discussed e.g. here .Therefore, the solution with reduce is cleaner.
Sounds like you need to evaluate the expressions in your list, and then reduce the resulting list by the addition function.
We can evaluate lisp expressions with eval, which we can apply to each element of the input list with mapcar.
We can then use reduce on the resulting list to find the sum.
(defun sum-expr (list)
(reduce #'+ (mapcar #'eval list)))
This makes a lot of assumptions about the structure and type of your input, but for a simple problem with well-understood inputs, it should be fine.
(You may be interested in Why exactly is eval evil?)

Generate code in macro lisp

I would like to generate code: (nth 0 x) (nth 1 x) ... (nth n x)
where x is just some variable name, and n is some number.
I'm trying do this in a following way:
(defmacro gen(n)
(loop for i from 1 to n do
`(nth i x))
)
Checking how it expands by:
(print (macroexpand-1 '(gen 5)))
Console output is: NIL. How to do it properly?
You need to replace do with collect in your loop.
Note however that your macro captures the variable x from the calling environment.
Generally speaking, "macros are advanced material", if you are not comfortable with loop, you probably should not be writing them.
Consider what the value of the following code is:
(loop for i from 1 to 5
do `(nth ,i x))
Since there's no collection happening, the return value of the loop is nil. If we change do to collect:
(loop for i from 1 to 5
collect `(nth ,i x))
We see that we are getting somewhere. However, the resulting list is not actually valid Common Lisp code (and relies on there being a variable x in the environment where the macro is used).
It is not clear what you want to do with these (just run them? they're side-effect free, so just wrapping this in a progn feels somewhat useless), but you need to either cons a progn, list or similar to the front of the list of lists to make it into valid code.
(defmacro gen (n &key (var 'x) (accumulator 'list))
(cons accumulator
(loop for i from 1 to n
collect `(nth ,i ,var))))
This eventually gives us this macro that seems to actually do something approaching "valid".

IF error in LISP

I am relatively new to Lisp and I was trying to do a linear search on LISP. But I haven't been able to do so. I am always getting an error that says that "IF has too few parameters".
(setq a '(8 6 2 3 9 5 1))
(LET (key))
(setq key (read))
(loop
(if(= (first a) (key)))
(return t)
(return NIL)
(setq a (rest a))
)
Many problems in your code:
Globally setq an undefined variable
(let (key)) alone does nothing. If you want to define a global variable, use defparameter or defvar.
You if has only a test, and no branches. The special operator if takes a condition, a then expression and an optional else expression: (if test then [else])
If you intended to have your return inside the if, your linear search would stop at the first comparison, because of (return NIL). Indeed, what you would have written would be equivalent to (return (= (first a) key)) and the loop would not even be needed in that case. Maybe you intended to use return to return a value from the if, but if is an expression an already evaluates as a value. return exits the loop (there is an implicit (block NIL ...) around the loop).
(setq a (rest a)) is like (pop a) and would indeed be the right thing to do if you did not already returned from loop at this point.
Just to be sure, be aware that = is for comparing numbers.
The beginning of your code can be written as:
(let ((a '(8 6 2 3 9 5 1))
(key (read)))
(linear-search key a)
Then, how you perform linear-search depends on what you want to learn. There are built-in for this (find, member). You can also use some with a predicate. Loop has a thereis clause. You can even try with reduce or map with a return-from. If you want to learn do or tagbody, you will have an occasion to use (pop a).

Is evaluating of constructed evaluation equal to macro?

I want to know if these two definitions of nth are equal:
I. is defined as macro:
(defmacro -nth (n lst)
(defun f (n1 lst1)
(cond ((eql n1 0) lst1)
(t `(cdr ,(f (- n1 1) lst1)))))
`(car ,(f n lst)))
II. is defined as a bunch of functions:
(defun f (n lst)
(cond ((eql n 0) lst)
(t `(cdr ,(f (- n 1) lst)))))
(defun f1 (n lst)
`(car ,(f n `',lst)))
(defun --nth (n lst)
(eval (f1 n lst)))
Am i get the right idea? Is macro definition is evaluating of expression, constructed in its body?
OK, let start from the beginning.
Macro is used to create new forms that usually depend on macro's input. Before code is complied or evaluated, macro has to be expanded. Expansion of a macro is a process that takes place before evaluation of form where it is used. Result of such expansion is usually a lisp form.
So inside a macro here are a several levels of code.
Not quoted code will be evaluated during macroexpansion (not at run-time!), in your example you define function f when macro is expanded (for what?);
Next here is quoted (with usual quote or backquote or even nested backquotes) code that will become part of macroexpansion result (in its literal form); you can control what part of code will be evaluated during macroexpansion and what will stay intact (quoted, partially or completely). This allows one to construct anything before it will be executed.
Another feature of macro is that it does not evaluate its parameters before expansion, while function does. To give you picture of what is a macro, see this (just first thing that came to mind):
(defmacro aif (test then &optional else)
`(let ((it ,test))
(if it ,then ,else)))
You can use it like this:
CL-USER> (defparameter *x* '((a . 1) (b . 2) (c . 3) (d . 4)))
*X*
CL-USER> (aif (find 'c *x* :key #'car) (1+ (cdr it)) 0)
4
This macro creates useful lexical binding, capturing variable it. After checking of a condition, you don't have to recalculate result, it's accessible in forms 'then' and 'else'. It's impossible to do with just a function, it has introduced new control construction in language. But macro is not just about creating lexical environments.
Macro is a powerful tool. It's impossible to fully describe what you can do with it, because you can do everything. But nth is not something you need a macro for. To construct a clone of nth you can try to write a recursive function.
It's important to note that LISP macro is most powerful thing in the programming world and LISP is the only language that has this power ;-)
To inspire you, I would recommend this article: http://www.paulgraham.com/avg.html
To master macro, begin with something like this:
http://www.gigamonkeys.com/book/macros-defining-your-own.html
Then may be Paul Graham's "On Lisp", then "Let Over Lambda".
There is no need for either a macro nor eval to make abstractions to get the nth element of a list. Your macro -nth doesn't even work unless the index is literal number. try this:
(defparameter test-list '(9 8 7 6 5 4 3 2 1 0))
(defparameter index 3)
(nth index test-list) ; ==> 6 (this is the LISP provided nth)
(-nth index test-list) ; ==> ERROR: index is not a number
A typical recursive solution of nth:
(defun nth2 (index list)
(if (<= index 0)
(car list)
(nth2 (1- index) (cdr list))))
(nth2 index test-list) ; ==> 6
A typical loop version
(defun nth3 (index list)
(loop :for e :in list
:for i :from index :downto 0
:when (= i 0) :return e))
(nth3 index test-list) ; ==> 6
Usually a macro is something you use when you see your are repeating yourself too much and there is no way to abstract your code further with functions. You may make a macro that saves you the time to write boilerplate code. Of course there is a trade off of not being standard code so you usually write the macro after a couple of times have written the boilerplate.
eval should never be used unless you really have to. Usually you can get by with funcall and apply. eval works only in the global scope so you loose closure variables.

scheme continuations for dummies

For the life of me, I can't understand continuations. I think the problem stems from the fact that I don't understand is what they are for. All the examples that I've found in books or online are very trivial. They make me wonder, why anyone would even want continuations?
Here's a typical impractical example, from TSPL, which I believe is quite recognized book on the subject. In english, they describe the continuation as "what to do" with the result of a computation. OK, that's sort of understandable.
Then, the second example given:
(call/cc
(lambda (k)
(* 5 (k 4)))) => 4
How does this make any sense?? k isn't even defined! How can this code be evaluated, when (k 4) can't even be computed? Not to mention, how does call/cc know to rip out the argument 4 to the inner most expression and return it? What happens to (* 5 .. ?? If this outermost expression is discarded, why even write it?
Then, a "less" trivial example stated is how to use call/cc to provide a nonlocal exit from a recursion. That sounds like flow control directive, ie like break/return in an imperative language, and not a computation.
And what is the purpose of going through these motions? If somebody needs the result of computation, why not just store it and recall later, as needed.
Forget about call/cc for a moment. Every expression/statement, in any programming language, has a continuation - which is, what you do with the result. In C, for example,
x = (1 + (2 * 3));
printf ("Done");
has the continuation of the math assignment being printf(...); the continuation of (2 * 3) is 'add 1; assign to x; printf(...)'. Conceptually the continuation is there whether or not you have access to it. Think for a moment what information you need for the continuation - the information is 1) the heap memory state (in general), 2) the stack, 3) any registers and 4) the program counter.
So continuations exist but usually they are only implicit and can't be accessed.
In Scheme, and a few other languages, you have access to the continuation. Essentially, behind your back, the compiler+runtime bundles up all the information needed for a continuation, stores it (generally in the heap) and gives you a handle to it. The handle you get is the function 'k' - if you call that function you will continue exactly after the call/cc point. Importantly, you can call that function multiple times and you will always continue after the call/cc point.
Let's look at some examples:
> (+ 2 (call/cc (lambda (cont) 3)))
5
In the above, the result of call/cc is the result of the lambda which is 3. The continuation wasn't invoked.
Now let's invoke the continuation:
> (+ 2 (call/cc (lambda (cont) (cont 10) 3)))
12
By invoking the continuation we skip anything after the invocation and continue right at the call/cc point. With (cont 10) the continuation returns 10 which is added to 2 for 12.
Now let's save the continuation.
> (define add-2 #f)
> (+ 2 (call/cc (lambda (cont) (set! add-2 cont) 3)))
5
> (add-2 10)
12
> (add-2 100)
102
By saving the continuation we can use it as we please to 'jump back to' whatever computation followed the call/cc point.
Often continuations are used for a non-local exit. Think of a function that is going to return a list unless there is some problem at which point '() will be returned.
(define (hairy-list-function list)
(call/cc
(lambda (cont)
;; process the list ...
(when (a-problem-arises? ...)
(cont '()))
;; continue processing the list ...
value-to-return)))
Here is text from my class notes: http://tmp.barzilay.org/cont.txt. It is based on a number of sources, and is much extended. It has motivations, basic explanations, more advanced explanations for how it's done, and a good number of examples that go from simple to advanced, and even some quick discussion of delimited continuations.
(I tried to play with putting the whole text here, but as I expected, 120k of text is not something that makes SO happy.
TL;DR: continuations are just captured GOTOs, with values, more or less.
The exampe you ask about,
(call/cc
(lambda (k)
;;;;;;;;;;;;;;;;
(* 5 (k 4)) ;; body of code
;;;;;;;;;;;;;;;;
)) => 4
can be approximately translated into e.g. Common Lisp, as
(prog (k retval)
(setq k (lambda (x) ;; capture the current continuation:
(setq retval x) ;; set! the return value
(go EXIT))) ;; and jump to exit point
(setq retval ;; get the value of the last expression,
(progn ;; as usual, in the
;;;;;;;;;;;;;;;;
(* 5 (funcall k 4)) ;; body of code
;;;;;;;;;;;;;;;;
))
EXIT ;; the goto label
(return retval))
This is just an illustration; in Common Lisp we can't jump back into the PROG tagbody after we've exited it the first time. But in Scheme, with real continuations, we can. If we set some global variable inside the body of function called by call/cc, say (setq qq k), in Scheme we can call it at any later time, from anywhere, re-entering into the same context (e.g. (qq 42)).
The point is, the body of call/cc form may contain an if or a condexpression. It can call the continuation only in some cases, and in others return normally, evaluating all expressions in the body of code and returning the last one's value, as usual. There can be deep recursion going on there. By calling the captured continuation an immediate exit is achieved.
So we see here that k is defined. It is defined by the call/cc call. When (call/cc g) is called, it calls its argument with the current continuation: (g the-current-continuation). the current-continuation is an "escape procedure" pointing at the return point of the call/cc form. To call it means to supply a value as if it were returned by the call/cc form itself.
So the above results in
((lambda(k) (* 5 (k 4))) the-current-continuation) ==>
(* 5 (the-current-continuation 4)) ==>
; to call the-current-continuation means to return the value from
; the call/cc form, so, jump to the return point, and return the value:
4
I won't try to explain all the places where continuations can be useful, but I hope that I can give brief examples of main place where I have found continuations useful in my own experience. Rather than speaking about Scheme's call/cc, I'd focus attention on continuation passing style. In some programming languages, variables can be dynamically scoped, and in languages without dynamically scoped, boilerplate with global variables (assuming that there are no issues of multi-threaded code, etc.) can be used. For instance, suppose there is a list of currently active logging streams, *logging-streams*, and that we want to call function in a dynamic environment where *logging-streams* is augmented with logging-stream-x. In Common Lisp we can do
(let ((*logging-streams* (cons logging-stream-x *logging-streams*)))
(function))
If we don't have dynamically scoped variables, as in Scheme, we can still do
(let ((old-streams *logging-streams*))
(set! *logging-streams* (cons logging-stream-x *logging-streams*)
(let ((result (function)))
(set! *logging-streams* old-streams)
result))
Now lets assume that we're actually given a cons-tree whose non-nil leaves are logging-streams, all of which should be in *logging-streams* when function is called. We've got two options:
We can flatten the tree, collect all the logging streams, extend *logging-streams*, and then call function.
We can, using continuation passing style, traverse the tree, gradually extending *logging-streams*, finally calling function when there is no more tree to traverse.
Option 2 looks something like
(defparameter *logging-streams* '())
(defun extend-streams (stream-tree continuation)
(cond
;; a null leaf
((null stream-tree)
(funcall continuation))
;; a non-null leaf
((atom stream-tree)
(let ((*logging-streams* (cons stream-tree *logging-streams*)))
(funcall continuation)))
;; a cons cell
(t
(extend-streams (car stream-tree)
#'(lambda ()
(extend-streams (cdr stream-tree)
continuation))))))
With this definition, we have
CL-USER> (extend-streams
'((a b) (c (d e)))
#'(lambda ()
(print *logging-streams*)))
=> (E D C B A)
Now, was there anything useful about this? In this case, probably not. Some minor benefits might be that extend-streams is tail-recursive, so we don't have a lot of stack usage, though the intermediate closures make up for it in heap space. We do have the fact that the eventual continuation is executed in the dynamic scope of any intermediate stuff that extend-streams set up. In this case, that's not all that important, but in other cases it can be.
Being able to abstract away some of the control flow, and to have non-local exits, or to be able to pick up a computation somewhere from a while back, can be very handy. This can be useful in backtracking search, for instance. Here's a continuation passing style propositional calculus solver for formulas where a formula is a symbol (a propositional literal), or a list of the form (not formula), (and left right), or (or left right).
(defun fail ()
'(() () fail))
(defun satisfy (formula
&optional
(positives '())
(negatives '())
(succeed #'(lambda (ps ns retry) `(,ps ,ns ,retry)))
(retry 'fail))
;; succeed is a function of three arguments: a list of positive literals,
;; a list of negative literals. retry is a function of zero
;; arguments, and is used to `try again` from the last place that a
;; choice was made.
(if (symbolp formula)
(if (member formula negatives)
(funcall retry)
(funcall succeed (adjoin formula positives) negatives retry))
(destructuring-bind (op left &optional right) formula
(case op
((not)
(satisfy left negatives positives
#'(lambda (negatives positives retry)
(funcall succeed positives negatives retry))
retry))
((and)
(satisfy left positives negatives
#'(lambda (positives negatives retry)
(satisfy right positives negatives succeed retry))
retry))
((or)
(satisfy left positives negatives
succeed
#'(lambda ()
(satisfy right positives negatives
succeed retry))))))))
If a satisfying assignment is found, then succeed is called with three arguments: the list of positive literals, the list of negative literals, and function that can retry the search (i.e., attempt to find another solution). For instance:
CL-USER> (satisfy '(and p (not p)))
(NIL NIL FAIL)
CL-USER> (satisfy '(or p q))
((P) NIL #<CLOSURE (LAMBDA #) {1002B99469}>)
CL-USER> (satisfy '(and (or p q) (and (not p) r)))
((R Q) (P) FAIL)
The second case is interesting, in that the third result is not FAIL, but some callable function that will try to find another solution. In this case, we can see that (or p q) is satisfiable by making either p or q true:
CL-USER> (destructuring-bind (ps ns retry) (satisfy '(or p q))
(declare (ignore ps ns))
(funcall retry))
((Q) NIL FAIL)
That would have been very difficult to do if we weren't using a continuation passing style where we can save the alternative flow and come back to it later. Using this, we can do some clever things, like collect all the satisfying assignments:
(defun satisfy-all (formula &aux (assignments '()) retry)
(setf retry #'(lambda ()
(satisfy formula '() '()
#'(lambda (ps ns new-retry)
(push (list ps ns) assignments)
(setf retry new-retry))
'fail)))
(loop while (not (eq retry 'fail))
do (funcall retry)
finally (return assignments)))
CL-USER> (satisfy-all '(or p (or (and q (not r)) (or r s))))
(((S) NIL) ; make S true
((R) NIL) ; make R true
((Q) (R)) ; make Q true and R false
((P) NIL)) ; make P true
We could change the loop a bit and get just n assignments, up to some n, or variations on that theme. Often times continuation passing style is not needed, or can make code hard to maintain and understand, but in the cases where it is useful, it can make some otherwise very difficult things fairly easy.