So I thought one of the advantages of lisp (among other languages) is its ability to implement function factories (accept functions as arguments; return new functions). I want to use this capability to make small changes to a function and save it as a new function so that if changes are made to the original function, they are also reflected in the new function on which it is based. Note: I am not the one writing the original function so I can't necessarily encapsulate the common parts in a separate function to be called by both, which would be the obvious answer otherwise.
Toy example in emacs lisp (may not be the most ideal as it is a lisp-2):
I have a function, foo that is provided to me:
(defun foo (x y)
(+ x y)))
I want my new function to include a statement that allows me to change the value of a variable if a certain condition is met. For instance:
(defun newfoo (x y)
(if (condition-met-p x)
(setq x (transform x)))
(+ x y))
Please disregard that I could use defadvice in this particular example as I am more interested in the general task of modifying functions where defadvice may not apply. I believe I can modify the body with this form:
(setq conditional-transformation
'(if (condition-met x) (setq x (transform x))))
(setq newbody (append conditional-transformation
(nth 2 (symbol-function 'foo)))))
My questions are specifically how to
create a copy of foo to newfoo
and replace the body with the value
of newbody defined above. (I've
looked into fset, setf, and
function but perhaps not using
them properly.)
possibly wrap this in a function
called makenewfoo() or something
like this so I can invoke
makenewfoo(foo) and allow this to
create newfoo().
And, more generally,
is something like this is commonly
done or there is a more idiomatic
way to modify functions?
this is a very simple case, but is
there a more general way than
specifying the list element number
to nth for the modification. For
instance, the actual function is
more complex so is there a way to
recursively search down this
s-expression tree and test for a
particular syntax and insert this
conditional-transformation
expression before or after it
(possibly using equal), so it is
less sensitive to changes made in
the original function?
It does work in Emacs Lisp:
elisp> (defun foo (x y)
(+ x y))
foo
elisp> (fset 'newfoo
(append (lambda (x y)
(when (< x 2)
(setq x (* x 2))))
(cddr (symbol-function 'foo))))
(lambda
(x y)
(when
(< x 2)
(setq x
(* x 2)))
(+ x y))
elisp> (newfoo 1 3)
5
elisp> (newfoo 3 3)
6
But I really don't think that it is commonly done or idiomatic. You should use defadvice if you want to modify the behavior of functions.
As far as CL is concerned: Some implementations provide similar functions/macros (for example in CCL: ccl:advise), and you can specify :before, :after, and :around methods for generic functions.
Example code for insertion of expressions:
(defun find-node (elt tree)
(cond ((null tree) nil)
((equal (car tree) elt) tree)
((consp (car tree)) (let ((node (find-node elt (car tree))))
(if node node (find-node elt (cdr tree)))))
(t (find-node elt (cdr tree)))))
(defun insert-before (node elt)
(setcdr node (cons (car node) (cdr node)))
(setcar node elt))
(let* ((function (copy-tree (symbol-function 'foo)))
(node (find-node '(+ x y) function)))
(when node
(insert-before node '(if (< x 2) (setq x (* x 2))))
(fset 'newfoo function)))
Related
I'm learning Lisp now, and I'm trying to do an exercise that asks me to get the maximum value of a list, the syntax is totally different from most programming languages I've learned, so I'm having some difficulties.
My code:
(defun test(y)
(cond
((and (first y) (> (second y)) (> (third y)))
(format t "numero maximo ~d" (first y))
((and (second y) (> (first y)) (> (third y)))
(t (format t "numero maximo ~d" (second y))
((and (third y) (> (second y)) (> (first y)))
(t (format t "numero maximo ~d" (third y))
))
I'm receiving this error: incomplete s-expression in region
Your code is too complex, it tries to take elements from a list, compare them, and print something. Like in other languages, use smaller functions and, particularly with a new language, test often in order to avoid having to debug something too large.
Your code, automatically indented with Emacs, looks as follows:
(defun test(y)
(cond
((and (first y) (> (second y)) (> (third y)))
(format t "numero maximo ~d" (first y))
((and (second y) (> (first y)) (> (third y)))
(t (format t "numero maximo ~d" (second y))
((and (third y) (> (second y)) (> (first y)))
(t (format t "numero maximo ~d" (third y))
))
And the editor complains about unbalanced parentheses:
In (> (second y)), the > function is given only one argument
All your cond clauses are in fact nested inside the first clause. Using an editor that highlights matching parentheses helps a lot here. The syntax should be:
(cond
(test-1 ...)
(test-2 ...)
(t ...))
If your test involves calling predicates, then it looks like:
(cond
((and (f1 ...) (f2 ...)) ;; <-- test
... ;; <-- code
) ;; end of first clause
) ;; end of cond
But note that you do not need to put comments for closing delimiters, the indentation and the automatic highlighting of parentheses should help you avoid mistakes.
Let's try a rewrite.
First of all, you can write a function that just compares numbers, not thinking about lists or formatting; here is a very straightforward max-of-3 implementation (without cheating and calling the built-in max function):
(defun max-of-3 (x y z)
(if (> x y)
(if (> x z) x z)
(if (> y z) y z)))
Evaluate the function, and test it on multiple inputs, for example in the REPL:
CL-USER> (max-of-3 0 2 1)
2
....
Then, you can build up the other function, for your list:
(defun test (list)
(format t
"numero maximo ~d"
(max-of-3 (first list)
(second list)
(third list))))
If you need to do more error checking ahead of time, like checking that the lists is well-formed, you should probably define other auxiliary functions.
If I understood the question and answer, I might be able to provide a solution or two that returns the max, regardless of the length of the list. So, these solutions are not limited to a list of three.
This illustrates a way to test where "max-lst" is the Lisp function under test:
(defconstant test-case
(list 1 2 0 8 7 6 9 4 5))
(defun run-test ()
(max-lst test-case))
Solution 1
This solution uses recursion. If you like loops better, Lisp has several loops. The Lisp function "max" is not used:
(defun max-lst (lst-in)
(cond ((null (second lst-in))
(first lst-in))
((> (first lst-in) (second lst-in))
(max-lst
(list* (first lst-in) (rest (rest lst-in)))))
(t
(max-lst
(list* (rest lst-in))))))
Solution 2
If you have no objection to using the Lisp function "max," here is a solution using max.
Note that max is not limited to two arguments.
(max 5 6 4 7 3)
will return 7.
In this solution, the function "max" is passed to the function "reduce" as an argument. The "reduce" function takes a function and a list as arguments. The function is applied to each adjacent pair of arguments and returns the result. If you want the sum, you can pass the + argument.
(defun max-lst-using-max (lst-in)
(reduce #'max lst-in))
Alas, I fear that I provide these solutions too late to be pertinent to the original poster. But maybe someone else will have a similar question. So, perhaps this could help, after all.
I'm trying to understand the following two snippets of code:
(defun make-adder1 (n) `(lambda (x) (+ ,n x)))
(defun make-adder2 (n) (lexical-let ((n n)) (lambda (x) (+ n x))))
These both seem to produce callables:
(funcall (make-adder1 3) 5) ;; returns 8
(funcall (make-adder2 3) 5) ;; returns 8
These both work. I have two main questions:
1) I don't understand the disparity in "quoting level" between the two approaches. In the first case, the lambda expression is quoted, which means the "symbol itself" is returned instead of the value. In the second case, it seems like the statement with the lambda will get evaluated, so the value of the lambda will be returned. Yet, these both work with funcall. When using funcall on a defun'ed function, it has to be quoted. Is lexical-let doing some kind of quoting automatically? Isn't this, kind of surprising?
2) Reading other posts on this topic, I'm given to understand that the first approach will break down under certain circumstances and deviate from what one would expect from working with lambdas and higher order functions in other languages, because elisp has dynamic scoping by default. Can someone give a concrete example of code that makes this difference apparent and explain it?
In the first example there is no variable n in the resulting function, which is just (lambda (x) (+ 3 x)). It does not need lexical binding because there is no free variable in the lambda, i.e., no variable that needs to be kept in a binding of a closure. If you don't need the variable n to be available, as a variable in uses of the function, i.e., if its value at function definition time (=3) is all you need, then the first example is all you need.
(fset 'ad1 (make-adder1 3))
(symbol-function 'ad1)
returns:
(lambda (x) (+ 3 x))
The second example creates what is, in effect, a function that creates and applies a complicated closure.
(fset 'ad2 (make-adder2 3))
(symbol-function 'ad2)
returns
(lambda (&rest --cl-rest--)
(apply (quote (closure ((--cl-n-- . --n--) (n . 3) t)
(G69710 x)
(+ (symbol-value G69710) x)))
(quote --n--)
--cl-rest--))
A third option is to use a lexical-binding file-local variable and use the most straightforward definition. This creates a simple closure.
;;; foo.el --- toto -*- lexical-binding: t -*-
(defun make-adder3 (n) (lambda (x) (+ n x)))
(fset 'ad3 (make-adder3 3))
(symbol-function 'ad3)
returns:
(closure ((n . 3) t) (x) (+ n x))
(symbol-function 'make-adder1)
returns:
(lambda (n)
(list (quote lambda)
(quote (x))
(cons (quote +) (cons n (quote (x))))))
(symbol-function 'make-adder2)
returns:
(closure (t)
(n)
(let ((--cl-n-- (make-symbol "--n--")))
(let* ((v --cl-n--)) (set v n))
(list (quote lambda)
(quote (&rest --cl-rest--))
(list (quote apply)
(list (quote quote)
(function
(lambda (G69709 x)
(+ (symbol-value G69709) x))))
(list (quote quote) --cl-n--)
(quote --cl-rest--)))))
(symbol-function 'make-adder3)
returns
(closure (t) (n) (function (lambda (x) (+ n x))))
In Section 12.4 of On Lisp, Paul Graham writes, "Unfortunately, we can't define a correct _f with define-modify-macro, because the operator to be applied to the generalized variable is given as an argument."
But what's wrong with something like this?
(define-modify-macro _f (op operand)
(lambda (x op operand)
(funcall op x operand)))
(let ((lst '(1 2 3)))
(_f (second lst) #'* 6)
lst)
=> (1 12 3)
Has there perhaps been a change made to define-modify-macro in ANSI Common Lisp that wasn't valid at the time On Lisp was written? Or are there reasons other than the one stated for not using define-modify-macro here?
It appears that Graham want's to be able to make a call such as
(_f * (second lst) 6)
rather than
(_f #'* (second lst) 6)
But surely that's not in keeping with a Lisp2 such as Common Lisp?
According to both Lispworks's Hyperspec and CLtL2 (look for define-modify-macro), the function is assumed to be a symbol (to a function or a macro). As far as I know, the following definition might not be conforming the specification:
(define-modify-macro _f (op operand)
(lambda (x op operand)
(funcall op x operand)))
But of course, it is possible that an implementation allows it.
To be sure you are conforming to the standard, you can define your own function, or even a macro:
(defmacro funcall-1 (val fun &rest args)
`(funcall ,fun ,val ,#args))
(define-modify-macro _ff (&rest args) funcall-1)
(let ((x (list 1 2 3 4)))
(_ff (third x) #'+ 10)
x)
If you wanted to have the function as a second argument, you could define another macro:
(defmacro ff (fun-form place &rest args)
`(_ff ,place ,fun-form ,#args))
Basically, your approach consists in wrapping funcall in define-modify-macro, and give the desired function as an argument of that function. At first sight, it looks like a hack, but as we can see below, this gives the same macroexanded code as the one in On Lisp, assuming we modify the latter a little.
The macroexpansion of the above is:
(LET ((X (LIST 1 2 3 4)))
(LET* ((#:G1164 X) (#:G1165 (FUNCALL #'+ (THIRD #:G1164) 10)))
(SB-KERNEL:%RPLACA (CDDR #:G1164) #:G1165))
X)
The version in On Lisp behaves as follows:
(defmacro _f (op place &rest args)
(multiple-value-bind (vars forms var set access)
(get-setf-expansion
place)
`(let* (,#(mapcar #'list vars forms)
(, (car var) (,op ,access ,#args)))
,set)))
(let ((x (list 1 2 3 4)))
(_f * (third x) 10)
x)
Macroexpansion:
(LET ((X (LIST 1 2 3 4)))
(LET* ((#:G1174 X) (#:G1175 (* (THIRD #:G1174) 10)))
(SB-KERNEL:%RPLACA (CDDR #:G1174) #:G1175))
X)
Here, the * is injected directly by the macroexpansion, which means that the resulting code has no possible runtime overhead (though compilers would probably handle your (funcall #'+ ...) equally well). If you pass #'+ to the macro, it fails to macroexpand. This is the major difference with your approach, but not a big limitation. In order to allow the On Lisp version to accept #'*, or even (create-closure) as an operator, it should be modified as follows:
(defmacro _f (op place &rest args)
(multiple-value-bind (vars forms var set access)
(get-setf-expansion
place)
`(let* (,#(mapcar #'list vars forms)
(, (car var) (funcall ,op ,access ,#args)))
,set)))
(see the call to funcall)
The previous example is then expanded as follows, for #'*:
(LET ((X (LIST 1 2 3 4)))
(LET* ((#:G1180 X) (#:G1181 (FUNCALL #'* (THIRD #:G1180) 10)))
(SB-KERNEL:%RPLACA (CDDR #:G1180) #:G1181))
X)
Now, it is exactly as your version. On Lisp uses _f to demonstrate how to use get-setf-expansion, and _f is a good example for that. But on the other hand, your implementation seems equally good.
On the question of whether one might prefer to pass * or #'*, we can also note that the define-modify-macro version of _f and #coredump's adapted version (with funcall) both accept lambda forms in the op position with or without #' e.g. both (lambda (x y) (* x y)) and #'(lambda (x y) (* x y)), whereas Graham's original version accepts only the former.
Interestingly in his book Let over Lambda, Doug Hoyte draws attention to a remark by Graham in his book ANSI Common Lisp that being able to omit the #' before a lambda form provides "a specious form of elegance at best" before going on to prefer to omit it.
I'm not taking a stand either way, merely pointing out that given Graham's choice for _f, the absence of the #' is no longer specious but necessary.
Could someone explain to me what's going on in this very simple code snippet?
(defun test-a ()
(let ((x '(nil)))
(setcar x (cons 1 (car x)))
x))
Upon a calling (test-a) for the first time, I get the expected result: ((1)).
But to my surprise, calling it once more, I get ((1 1)), ((1 1 1)) and so on.
Why is this happening? Am I wrong to expect (test-a) to always return ((1))?
Also note that after re-evaluating the definition of test-a, the return result resets.
Also consider that this function works as I expect:
(defun test-b ()
(let ((x '(nil)))
(setq x (cons (cons 1 (car x))
(cdr x)))))
(test-b) always returns ((1)).
Why aren't test-a and test-b equivalent?
The Bad
test-a is self-modifying code. This is extremely dangerous. While the variable x disappears at the end of the let form, its initial value persists in the function object, and that is the value you are modifying. Remember that in Lisp a function is a first class object, which can be passed around (just like a number or a list), and, sometimes, modified. This is exactly what you are doing here: the initial value for x is a part of the function object and you are modifying it.
Let us actually see what is happening:
(symbol-function 'test-a)
=> (lambda nil (let ((x (quote (nil)))) (setcar x (cons 1 (car x))) x))
(test-a)
=> ((1))
(symbol-function 'test-a)
=> (lambda nil (let ((x (quote ((1))))) (setcar x (cons 1 (car x))) x))
(test-a)
=> ((1 1))
(symbol-function 'test-a)
=> (lambda nil (let ((x (quote ((1 1))))) (setcar x (cons 1 (car x))) x))
(test-a)
=> ((1 1 1))
(symbol-function 'test-a)
=> (lambda nil (let ((x (quote ((1 1 1))))) (setcar x (cons 1 (car x))) x))
The Good
test-b returns a fresh cons cell and thus is safe. The initial value of x is never modified. The difference between (setcar x ...) and (setq x ...) is that the former modifies the object already stored in the variable x while the latter stores a new object in x. The difference is similar to x.setField(42) vs. x = new MyObject(42) in C++.
The Bottom Line
In general, it is best to treat quoted data like '(1) as constants - do not modify them:
quote returns the argument, without evaluating it. (quote x) yields x.
Warning: quote does not construct its return value, but just returns
the value that was pre-constructed by the Lisp reader (see info node
Printed Representation). This means that (a . b) is not
identical to (cons 'a 'b): the former does not cons. Quoting should
be reserved for constants that will never be modified by side-effects,
unless you like self-modifying code. See the common pitfall in info
node Rearrangement for an example of unexpected results when
a quoted object is modified.
If you need to modify a list, create it with list or cons or copy-list instead of quote.
See more examples.
PS1. This has been duplicated on Emacs.
PS2. See also Why does this function return a different value every time? for an identical Common Lisp issue.
PS3. See also Issue CONSTANT-MODIFICATION.
I found the culprit is indeed 'quote. Here's its doc-string:
Return the argument, without evaluating it.
...
Warning: `quote' does not construct its return value, but just returns
the value that was pre-constructed by the Lisp reader
...
Quoting should be reserved for constants that will
never be modified by side-effects, unless you like self-modifying code.
I also rewrote for convenience
(setq test-a
(lambda () ((lambda (x) (setcar x (cons 1 (car x))) x) (quote (nil)))))
and then used
(funcall test-a)
to see how 'test-a was changing.
It looks like the '(nil) in your (let) is only evaluated once. When you (setcar), each call is modifying the same list in-place. You can make (test-a) work if you replace the '(nil) with (list (list)), although I presume there's a more elegant way to do it.
(test-b) constructs a totally new list from cons cells each time, which is why it works differently.
I'm a Lisp beginner. I'm trying to memoize a recursive function for calculating the number of terms in a Collatz sequence (for problem 14 in Project Euler). My code as of yet is:
(defun collatz-steps (n)
(if (= 1 n) 0
(if (evenp n)
(1+ (collatz-steps (/ n 2)))
(1+ (collatz-steps (1+ (* 3 n)))))))
(defun p14 ()
(defvar m-collatz-steps (memoize #'collatz-steps))
(let
((maxsteps (funcall m-collatz-steps 2))
(n 2)
(steps))
(loop for i from 1 to 1000000
do
(setq steps (funcall m-collatz-steps i))
(cond
((> steps maxsteps)
(setq maxsteps steps)
(setq n i))
(t ())))
n))
(defun memoize (fn)
(let ((cache (make-hash-table :test #'equal)))
#'(lambda (&rest args)
(multiple-value-bind
(result exists)
(gethash args cache)
(if exists
result
(setf (gethash args cache)
(apply fn args)))))))
The memoize function is the same as the one given in the On Lisp book.
This code doesn't actually give any speedup compared to the non-memoized version. I believe it's due to the recursive calls calling the non-memoized version of the function, which sort of defeats the purpose. In that case, what is the correct way to do the memoization here? Is there any way to have all calls to the original function call the memoized version itself, removing the need for the special m-collatz-steps symbol?
EDIT: Corrected the code to have
(defvar m-collatz-steps (memoize #'collatz-steps))
which is what I had in my code.
Before the edit I had erroneously put:
(defvar collatz-steps (memoize #'collatz-steps))
Seeing that error gave me another idea, and I tried using this last defvar itself and changing the recursive calls to
(1+ (funcall collatz-steps (/ n 2)))
(1+ (funcall collatz-steps (1+ (* 3 n))))
This does seem to perform the memoization (speedup from about 60 seconds to 1.5 seconds), but requires changing the original function. Is there a cleaner solution which doesn't involve changing the original function?
I assume you're using Common-Lisp, which has separate namespaces for variable and function names. In order to memoize the function named by a symbol, you need to change its function binding, through the accessor `fdefinition':
(setf (fdefinition 'collatz-steps) (memoize #'collatz-steps))
(defun p14 ()
(let ((mx 0) (my 0))
(loop for x from 1 to 1000000
for y = (collatz-steps x)
when (< my y) do (setf my y mx x))
mx))
Here is a memoize function that rebinds the symbol function:
(defun memoize-function (function-name)
(setf (symbol-function function-name)
(let ((cache (make-hash-table :test #'equal)))
#'(lambda (&rest args)
(multiple-value-bind
(result exists)
(gethash args cache)
(if exists
result
(setf (gethash args cache)
(apply fn args)))))))
You would then do something like this:
(defun collatz-steps (n)
(if (= 1 n) 0
(if (evenp n)
(1+ (collatz-steps (/ n 2)))
(1+ (collatz-steps (1+ (* 3 n)))))))
(memoize-function 'collatz-steps)
I'll leave it up to you to make an unmemoize-function.
something like this:
(setf collatz-steps (memoize lambda (n)
(if (= 1 n) 0
(if (evenp n)
(1+ (collatz-steps (/ n 2)))
(1+ (collatz-steps (1+ (* 3 n))))))))
IOW: your original (non-memoized) function is anonymous, and you only give a name to the result of memoizing it.
Note a few things:
(defun foo (bar)
... (foo 3) ...)
Above is a function that has a call to itself.
In Common Lisp the file compiler can assume that FOO does not change. It will NOT call an updated FOO later. If you change the function binding of FOO, then the call of the original function will still go to the old function.
So memoizing a self recursive function will NOT work in the general case. Especially not if you are using a good compiler.
You can work around it to go always through the symbol for example: (funcall 'foo 3)
(DEFVAR ...) is a top-level form. Don't use it inside functions. If you have declared a variable, set it with SETQ or SETF later.
For your problem, I'd just use a hash table to store the intermediate results.
Changing the "original" function is necessary, because, as you say, there's no other way for the recursive call(s) to be updated to call the memoized version.
Fortunately, the way lisp works is to find the function by name each time it needs to be called. This means that it is sufficient to replace the function binding with the memoized version of the function, so that recursive calls will automatically look up and reenter through the memoization.
huaiyuan's code shows the key step:
(setf (fdefinition 'collatz-steps) (memoize #'collatz-steps))
This trick also works in Perl. In a language like C, however, a memoized version of a function must be coded separately.
Some lisp implementations provide a system called "advice", which provides a standardized structure for replacing functions with enhanced versions of themselves. In addition to functional upgrades like memoization, this can be extremely useful in debugging by inserting debug prints (or completely stopping and giving a continuable prompt) without modifying the original code.
This function is exactly the one Peter Norvig gives as an example of a function that seems like a good candidate for memoization, but which is not.
See figure 3 (the function 'Hailstone') of his original paper on memoization ("Using Automatic Memoization as a Software Engineering Tool in Real-World AI Systems").
So I'm guessing, even if you get the mechanics of memoization working, it won't really speed it up in this case.
A while ago I wrote a little memoization routine for Scheme that used a chain of closures to keep track of the memoized state:
(define (memoize op)
(letrec ((get (lambda (key) (list #f)))
(set (lambda (key item)
(let ((old-get get))
(set! get (lambda (new-key)
(if (equal? key new-key) (cons #t item)
(old-get new-key))))))))
(lambda args
(let ((ans (get args)))
(if (car ans) (cdr ans)
(let ((new-ans (apply op args)))
(set args new-ans)
new-ans))))))
This needs to be used like so:
(define fib (memoize (lambda (x)
(if (< x 2) x
(+ (fib (- x 1)) (fib (- x 2)))))))
I'm sure that this can be ported to your favorite lexically scoped Lisp flavor with ease.
I'd probably do something like:
(let ((memo (make-hash-table :test #'equal)))
(defun collatz-steps (n)
(or (gethash n memo)
(setf (gethash n memo)
(cond ((= n 1) 0)
((oddp n) (1+ (collatz-steps (+ 1 n n n))))
(t (1+ (collatz-steps (/ n 2)))))))))
It's not Nice and Functional, but, then, it's not much hassle and it does work. Downside is that you don't get a handy unmemoized version to test with and clearing the cache is bordering on "very difficult".