Could someone explain to me what's going on in this very simple code snippet?
(defun test-a ()
(let ((x '(nil)))
(setcar x (cons 1 (car x)))
x))
Upon a calling (test-a) for the first time, I get the expected result: ((1)).
But to my surprise, calling it once more, I get ((1 1)), ((1 1 1)) and so on.
Why is this happening? Am I wrong to expect (test-a) to always return ((1))?
Also note that after re-evaluating the definition of test-a, the return result resets.
Also consider that this function works as I expect:
(defun test-b ()
(let ((x '(nil)))
(setq x (cons (cons 1 (car x))
(cdr x)))))
(test-b) always returns ((1)).
Why aren't test-a and test-b equivalent?
The Bad
test-a is self-modifying code. This is extremely dangerous. While the variable x disappears at the end of the let form, its initial value persists in the function object, and that is the value you are modifying. Remember that in Lisp a function is a first class object, which can be passed around (just like a number or a list), and, sometimes, modified. This is exactly what you are doing here: the initial value for x is a part of the function object and you are modifying it.
Let us actually see what is happening:
(symbol-function 'test-a)
=> (lambda nil (let ((x (quote (nil)))) (setcar x (cons 1 (car x))) x))
(test-a)
=> ((1))
(symbol-function 'test-a)
=> (lambda nil (let ((x (quote ((1))))) (setcar x (cons 1 (car x))) x))
(test-a)
=> ((1 1))
(symbol-function 'test-a)
=> (lambda nil (let ((x (quote ((1 1))))) (setcar x (cons 1 (car x))) x))
(test-a)
=> ((1 1 1))
(symbol-function 'test-a)
=> (lambda nil (let ((x (quote ((1 1 1))))) (setcar x (cons 1 (car x))) x))
The Good
test-b returns a fresh cons cell and thus is safe. The initial value of x is never modified. The difference between (setcar x ...) and (setq x ...) is that the former modifies the object already stored in the variable x while the latter stores a new object in x. The difference is similar to x.setField(42) vs. x = new MyObject(42) in C++.
The Bottom Line
In general, it is best to treat quoted data like '(1) as constants - do not modify them:
quote returns the argument, without evaluating it. (quote x) yields x.
Warning: quote does not construct its return value, but just returns
the value that was pre-constructed by the Lisp reader (see info node
Printed Representation). This means that (a . b) is not
identical to (cons 'a 'b): the former does not cons. Quoting should
be reserved for constants that will never be modified by side-effects,
unless you like self-modifying code. See the common pitfall in info
node Rearrangement for an example of unexpected results when
a quoted object is modified.
If you need to modify a list, create it with list or cons or copy-list instead of quote.
See more examples.
PS1. This has been duplicated on Emacs.
PS2. See also Why does this function return a different value every time? for an identical Common Lisp issue.
PS3. See also Issue CONSTANT-MODIFICATION.
I found the culprit is indeed 'quote. Here's its doc-string:
Return the argument, without evaluating it.
...
Warning: `quote' does not construct its return value, but just returns
the value that was pre-constructed by the Lisp reader
...
Quoting should be reserved for constants that will
never be modified by side-effects, unless you like self-modifying code.
I also rewrote for convenience
(setq test-a
(lambda () ((lambda (x) (setcar x (cons 1 (car x))) x) (quote (nil)))))
and then used
(funcall test-a)
to see how 'test-a was changing.
It looks like the '(nil) in your (let) is only evaluated once. When you (setcar), each call is modifying the same list in-place. You can make (test-a) work if you replace the '(nil) with (list (list)), although I presume there's a more elegant way to do it.
(test-b) constructs a totally new list from cons cells each time, which is why it works differently.
Related
I want to implement the sorting function in common-lisp with this INSERT function
k means cons cell with number & val, and li means list where I want insert k into.
with this function, I can make a list of cell
(defun INSERT (li k) (IF (eq li nil) (cons (cons(car k)(cdr k)) nil)
(IF (eq (cdr li) nil)
(IF (< (car k)(caar li)) (cons (cons(car k)(cdr k)) li)
(cons (car li) (cons (cons(car k)(cdr k)) (cdr li)) )
)
(cond
( (eq (< (caar li) (car k)) (< (car k) (caadr li)) )
(cons (car k) (cons (cons (car k) (cdr k)) (cdr li)) ) )
(t (cons (car li) (INSERT (cdr li) k)) )))))
and what I want is the code of this function below. it has only one parameter li(non sorted list)
(defun Sort_List (li)(...this part...))
without using assignment, and using the INSERT function
Your insert function is very strange. In fact I find it so hard to read that I cn't work out what it's doing except that there's no need to check for both the list being null and its cdr being null. It also conses a lot of things it doesn't need, unless you are required by some part of the specification of the problem to make copies of the conses you are inserting.
Here is a version of it which is much easier to read and which does not copy when it does not need to. Note that this takes its arguments in the other order to yours:
(defun insert (thing into)
(cond ((null into)
(list thing))
((< (car thing) (car (first into)))
(cons thing into))
(t (cons (first into)
(insert thing (rest into))))))
Now, what is the algorithm for insertion sort? Well, essentially it is:
loop over the list to be sorted:
for each element, insert it into the sorted list;
finally return the sorted list.
And we're not allowed to use assignment to do this.
Well, there is a standard trick to do this sort of thing, which is to use a tail-recursive function with an accumulator argument, which accumulates the results. We can either write this function as an explicit auxiliary function, or we can make it a local function. I'm going to do the latter both because there's no reason for a function which is only ever used locally to be globally visible, and because (as I'm assuming this is homework) it makes it harder to submit directly.
So here is this version of the function:
(defun insertion-sort (l)
(labels ((is-loop (tail sorted)
(if (null tail)
sorted
(is-loop (rest tail) (insert (first tail) sorted)))))
(is-loop l '())))
This approach is fairly natural in Scheme, but not very natural in CL. An alternative approach which does not use assignment, at least explicitly, is to use do. Here is a version which uses do:
(defun insertion-sort (l)
(do ((tail l (rest tail))
(sorted '() (insert (first tail) sorted)))
((null tail) sorted)))
There are two notes about this version.
First of all, although it's not explicitly using assignment it pretty clearly implicitly is doing so. I think that's probably cheating.
Secondly it's a bit subtle why it works: what, exactly, is the value of tail in (insert (first tail) sorted), and why?
A version which is clearer, but uses loop which you are probably not meant to know about, is
(defun insertion-sort (l)
(loop for e in l
for sorted = (insert e '()) then (insert e sorted)
finally (return sorted)))
This, however, is also pretty explicitly using assignment.
As Kaz has pointed out below, there is an obvious way (which I should have seen!) of doing this using the CL reduce function. What reduce does, conceptually, is to successively collapse a sequence of elements by calling a function which takes two arguments. So, for instance
(reduce #'+ '(1 2 3 4))
is the same as
(+ (+ (+ 1 2) 3) 4)
This is easier to see if you use cons as the function:
> > (reduce #'cons '(1 2 3 4))
(((1 . 2) . 3) . 4)
> (cons (cons (cons 1 2) 3) 4)
(((1 . 2) . 3) . 4)
Well, of course, insert, as defined above, is really suitable for this: it takes an ordered list and inserts a new pair into it, returning a new ordered list. There are two problems:
my insert takes its arguments in the wrong order (this is possibly why the original one took the arguments in the other order!);
there needs to be a way of 'seeding' the initial sorted list, which will be ().
Well we can fix the wrong-argument-order either by rewriting insert, or just by wrapping it in a function which swaps the arguments: I'll do the latter because I don't want to revisit what I wrote above and I don't want two versions of the function.
You can 'seed' the initial null value by either just prepending it to the list of things to sort, or in fact reduce has a special option to provide the initial value, so we'll use that.
So using reduce we get this version of insertion-sort:
(defun insertion-sort (l)
(reduce (lambda (a e)
(insert e a))
l :initial-value '()))
And we can test this:
> (insertion-sort '((1 . a) (-100 . 2) (64.2 . "x") (-2 . y)))
((-100 . 2) (-2 . y) (1 . a) (64.2 . "x"))
and it works fine.
So the final question the is: are we yet again cheating by using some function whose definition obviously must involve assignment? Well, no, we're not, because you can quite easily write a simplified reduce and see that it does not need to use assignment. This version is much simpler than CL's reduce, and in particular it explicitly requires the initial-value argument:
(defun reduce/simple (f list accum)
(if (null list)
accum
(reduce/simple f (rest list) (funcall f accum (first list)))))
(Again, this is not very natural CL code since it relies on tail-call elimination to handle large lists, but it makes the point that you can do this without assignment.)
And so now we can write one final version of insertion-sort:
(defun insertion-sort (l)
(reduce/simple (lambda (a e)
(insert e a))
l '()))
And it's easy to check that this works as well.
I have the following setup in Common Lisp. my-object is a list of 5 binary trees.
(defun make-my-object ()
(loop for i from 0 to 5
for nde = (init-tree)
collect nde))
Each binary tree is a list of size 3 with a node, a left child and a right child
(defstruct node
(min 0)
(max 0)
(ctr 0))
(defun vals (tree)
(car tree))
(defun left-branch (tree)
(cadr tree))
(defun right-branch (tree)
(caddr tree))
(defun make-tree (vals left right)
(list vals left right))
(defun init-tree (&key (min 0) (max 1))
(let ((n (make-node :min min :max max)))
(make-tree n '() '())))
Now, I was trying to add an element to one of the binary trees manually, like this:
(defparameter my-object (make-my-object))
(print (left-branch (car my-object))) ;; returns NIL
(let ((x (left-branch (car my-object))))
(setf x (cons (init-tree) x)))
(print (left-branch (car my-object))) ;; still returns NIL
The second call to print still returns NIL. Why is this? How can I add an element to the binary tree?
The first function is just:
(defun make-my-object ()
(loop repeat 5 collect (init-tree)))
Now you define a structure for node, but you use a list for the tree and my-object? Why aren't they structures?
Instead of car, cadr and caddr one would use first, second, third.
(let ((x (left-branch (car my-object))))
(setf x (cons (init-tree) x)))
You set the local variable x to a new value. Why? After the let the local variable is also gone. Why aren't you setting the left branch instead? You would need to define a way to do so. Remember: Lisp functions return values, not memory locations you can later set. How can you change the contents in a list? Even better: use structures and change the slot value. The structure (or even CLOS classes) has following advantages over plain lists: objects carry a type, slots are named, accessors are created, a make function is created, a type predicate is created, ...
Anyway, I would define structures or CLOS classes for node, tree and object...
Most of the code in this question isn't essential to the real problem here. The real problem comes in with the misunderstanding of this code:
(let ((x (left-branch (car my-object))))
(setf x (cons (init-tree) x)))
We can see the same kind of behavior without user-defined structures of any kind:
(let ((cell (cons 1 2)))
(print cell) ; prints (1 . 2)
(let ((x (car cell)))
(setf x 3)
(print cell))) ; prints (1 . 2)
If you understand why both print statements produce (1 . 2), then you've got enough to understand why your own code isn't doing what you (previously) expected it to do.
There are two variables in play here: cell and x. There are three values that we're concerned with 1, 2, and the cons-cell produced by the call (cons 1 2). Variables in Lisp are often called bindings; the variable, or name, is bound to a value. The variable cell is bound to the the cons cell (1 . 2). When we go into the inner let, we evaluate (car cell) to produce the value 1, which is then bound to the variable x. Then, we assign a new value, 3, to the variable x. That doesn't modify the cons cell that contains the value that x was originally bound to. Indeed, the value that was originally bound to x was produced by (car cell), and once the call to (car cell) returned, the only value that mattered was 1.
If you have some experience in other programming languages, this is directly analogous to something like
int[] array = ...;
int x = array[2]; // read from the array; assign result to x
x = 42; // doesn't modify the array
If you want to modify a structure, you need to setf the appropriate part of the structure. E.g.:
(let ((cell (cons 1 2)))
(print cell) ; prints (1 . 2)
(setf (car cell) 3)
(print cell)) ; prints (3 . 2)
I'm trying different binding models for macro lambda lists.
Edit: in fact the lambda list for my test macros is always (&rest ...). Which means that I'm 'destructuring' the argument list and not the lambda list. I try to get a solution that works for combining optional with key arguments or rest/body with key arguments - both combinations don't work in the Common Lisp standard implementation.
So I have different functions giving me a list of bindings having the same syntax as used by 'let'.
E.g:
(build-bindings ...) => ((first 1) middle (last "three"))
Now I thought to use a simple macro inside my test macros feeding such a list to 'let'.
This is trivial if I have a literal list:
(defmacro let-list (_list &rest _body)
`(let ,_list ,#_body))
(let-list ((a 236)) a) => 236
But that's the same as a plain 'let'.
What I'd like to have is the same thing with a generated list.
So e.g.
(let-list (build-bindings ...)
(format t "first: ~s~%" first)
last)
with (build-bindings ...), evaluated in the same lexical scope as the call (let-list ...), returning
((first 1) middle (last "three"))
the expansion of the macro should be
(let
((first 1) middle (last "three"))
(format t "first: ~s~%" first)
last)
and should print 1 and return "three".
Any idea how to accomplish that?
Edit (to make the question more general):
If I have a list of (symbol value) pairs, i.e. same syntax that let requires for it's list of bindings, e.g. ((one 1) (two 'two) (three "three")), is there any way to write a macro that creates lexical bindings of the symbols with the supplied values for it's &rest/&body parameter?
This is seems to be a possible solution which Joshua pointed me to:
(let ((list_ '((x 23) (y 6) z)))
(let
((symbols_(loop for item_ in list_
collect (if (listp item_) (car item_) item_)))
(values_ (loop for item_ in list_
collect (if (listp item_) (cadr item_) nil))))
(progv symbols_ values_
(format t "x ~s, y ~s, z ~s~%" x y z))))
evaluates to:
;Compiler warnings :
; In an anonymous lambda form: Undeclared free variable X
; In an anonymous lambda form: Undeclared free variable Y
; In an anonymous lambda form: Undeclared free variable Z
x 23, y 6, z NIL
I could also easily rearrange my build-bindings functions to return the two lists needed.
One problem is, that the compiler spits warnings if the variables have never been declared special.
And the other problem that, if the dynamically bound variables are also used in a surrounding lexical binding, they a shadowed by the lexical binding - again if they have never been declared special:
(let ((x 47) (y 11) (z 0))
(let ((list_ '((x 23) (y 6) z)))
(let
((symbols_(loop for item_ in list_
collect (if (listp item_) (car item_) item_)))
(values_ (loop for item_ in list_
collect (if (listp item_) (cadr item_) nil))))
(progv symbols_ values_
(format t "x ~s, y ~s, z ~s~%" x y z)))))
evaluates to:
x 47, y 11, z 0
A better way could be:
(let ((x 47) (y 11) (z 0))
(locally
(declare (special x y))
(let ((list_ '((x 23) (y 6) z)))
(let
((symbols_(loop for item_ in list_
collect (if (listp item_) (car item_) item_)))
(values_ (loop for item_ in list_
collect (if (listp item_) (cadr item_) nil))))
(progv symbols_ values_
(format t "x ~s, y ~s, z ~s~%" x y z))))))
evaluates to:
;Compiler warnings about unused lexical variables skipped
x 23, y 6, z NIL
I can't see at the moment whether there are other problems with the dynamic progv bindings.
But the whole enchilada of a progv wrapped in locally with all the symbols declared as special cries for a macro again - which is again not possible due to same reasons let-list doesn't work :(
The possiblilty would be a kind of macro-lambda-list destructuring-hook which I'm not aware of.
I have to look into the implementation of destructuring-bind since that macro does kind of what I'd like to do. Perhaps that will enlight me ;)
So a first (incorrect) attempt would look something like this:
(defun build-bindings ()
'((first 1) middle (last "three")))
(defmacro let-list (bindings &body body)
`(let ,bindings
,#body))
Then you could try doing something like:
(let-list (build-bindings)
(print first))
That won't work, of course, because the macro expansion leaves the form (build-bindings) in the resulting let, in a position where it won't be evaluated:
CL-USER> (pprint (macroexpand-1 '(let-list (build-bindings)
(print first))))
(LET (BUILD-BINDINGS)
(PRINT FIRST))
Evaluation during Macroexpansion time
The issue is that you want the result of build-bindings at macroexpansion time, and that's before the code as a whole is run. Now, in this example, build-bindings can be run at macroexpansion time, because it's not doing anything with any arguments (remember I asked in a comment what the arguments are?). That means that you could actually eval it in the macroexpansion:
(defmacro let-list (bindings &body body)
`(let ,(eval bindings)
,#body))
CL-USER> (pprint (macroexpand-1 '(let-list (build-bindings)
(print first))))
(LET ((FIRST 1) MIDDLE (LAST "three"))
(PRINT FIRST))
Now that will work, insofar as it will bind first, middle, and last to 1, nil, and "three", respectively. However, if build-bindings actually needed some arguments that weren't available at macroexpansion time, you'd be out of luck. First, it can take arguments that are available at macroexpansion time (e.g., constants):
(defun build-bindings (a b &rest cs)
`((first ',a) (middle ',b) (last ',cs)))
CL-USER> (pprint (macroexpand-1 '(let-list (build-bindings 1 2 3 4 5)
(print first))))
(LET ((FIRST '1) (MIDDLE '2) (LAST '(3 4 5)))
(PRINT FIRST))
You could also have some of the variables appear in there:
(defun build-bindings (x ex y why)
`((,x ,ex) (,y ,why)))
CL-USER> (pprint (macroexpand-1 '(let-list (build-bindings 'a 'ay 'b 'bee)
(print first))))
(LET ((A AY) (B BEE))
(PRINT FIRST))
What you can't do, though, is have the variable names be determined from values that don't exist until runtime. E.g., you can't do something like:
(let ((var1 'a)
(var2 'b))
(let-list (build-bindings var1 'ay var2 'bee)
(print first))
because (let-list (build-bindings …) …) is macroexpanded before any of this code is actually executed. That means that you'd be trying to evaluate (build-bindings var1 'ay var2 'bee) when var1 and var2 aren't bound to any values.
Common Lisp does all its macroexpansion first, and then evaluates code. That means that values that aren't available until runtime are not available at macroexpansion time.
Compilation (and Macroexpansion) at Runtime
Now, even though I said that Common Lisp does all its macroexpansion first, and then evaluates code, the code above actually uses eval at macroexpansion to get some extra evaluation earlier. We can do things in the other direction too; we can use compile at runtime. That means that we can generate a lambda function and compile it based on code (e.g., variable names) provided at runtime. We can actually do this without using a macro:
(defun %dynamic-lambda (bindings body)
(flet ((to-list (x) (if (listp x) x (list x))))
(let* ((bindings (mapcar #'to-list bindings))
(vars (mapcar #'first bindings))
(vals (mapcar #'second bindings)))
(apply (compile nil `(lambda ,vars ,#body)) vals))))
CL-USER> (%dynamic-lambda '((first 1) middle (last "three"))
'((list first middle last)))
;=> (1 NIL "three")
This compiles a lambda expression that is created at runtime from a body and a list of bindings. It's not hard to write a macro that takes some fo the quoting hassle out of the picture:
(defmacro let-list (bindings &body body)
`(%dynamic-lambda ,bindings ',body))
CL-USER> (let-list '((first 1) middle (last "three"))
(list first middle last))
;=> (1 NIL "three")
CL-USER> (macroexpand-1 '(let-list (build-bindings)
(list first middle last)))
;=> (%DYNAMIC-LAMBDA (BUILD-BINDINGS) '((LIST FIRST MIDDLE LAST)))
CL-USER> (flet ((build-bindings ()
'((first 1) middle (last "three"))))
(let-list (build-bindings)
(list first middle last)))
;=> (1 NIL "three")
This gives you genuine lexical variables from a binding list created at runtime. Of course, because the compilation is happening at runtime, you lose access to the lexical environment. That means that the body that you're compiling into a function cannot access the "surrounding" lexical scope. E.g.:
CL-USER> (let ((x 3))
(let-list '((y 4))
(list x y)))
; Evaluation aborted on #<UNBOUND-VARIABLE X {1005B6C2B3}>.
Using PROGV and special variables
If you don't need lexical variables, but can use special (i.e., dynamically scoped) variables instead, you can establish bindings at runtime using progv. That would look something like:
(progv '(a b c) '(1 2 3)
(list c b a))
;;=> (3 2 1)
You'll probably get some warnings with that if run it, because when the form is compiled, there's no way to know that a, b, and c are supposed to be special variables. You can use locally to add some special declarations, though:
(progv '(a b c) '(1 2 3)
(locally
(declare (special a b c))
(list c b a)))
;;=> (3 2 1)
Of course, if you're doing this, then you have to know the variables in advance which is exactly what you were trying to avoid in the first place. However, if you're willing to know the names of the variables in advance (and your comments seem like you might be okay with that), then you can actually use lexical variables.
Lexical variables with values computed at run time
If you're willing to state what the variables will be, but still want to compute their values dynamically at run time, you can do that relatively easily. First, lets write the direct version (with no macro):
;; Declare three lexical variables, a, b, and c.
(let (a b c)
;; Iterate through a list of bindings (as for LET)
;; and based on the name in the binding, assign the
;; corresponding value to the lexical variable that
;; is identified by the same symbol in the source:
(dolist (binding '((c 3) (a 1) b))
(destructuring-bind (var &optional value)
(if (listp binding) binding (list binding))
(ecase var
(a (setf a value))
(b (setf b value))
(c (setf c value)))))
;; Do something with the lexical variables:
(list a b c))
;;=> (1 NIL 3)
Now, it's not too hard to write a macrofied version of this. This version isn't perfect, (e.g., there could be hygiene issues with names, and declarations in the body won't work (because the body is being spliced in after some stuff). It's a start, though:
(defmacro computed-let (variables bindings &body body)
(let ((assign (gensym (string '#:assign-))))
`(let ,variables
(flet ((,assign (binding)
(destructuring-bind (variable &optional value)
(if (listp binding) binding (list binding))
(ecase variable
,#(mapcar (lambda (variable)
`(,variable (setf ,variable value)))
variables)))))
(map nil #',assign ,bindings))
,#body)))
(computed-let (a b c) '((a 1) b (c 3))
(list a b c))
;;=> (1 NIL 3)
One way of making this cleaner would be to avoid the assignment altogether, and the computed values to provide the values for the binding directly:
(defmacro computed-let (variables bindings &body body)
(let ((values (gensym (string '#:values-)))
(variable (gensym (string '#:variable-))))
`(apply #'(lambda ,variables ,#body)
(let ((,values (mapcar #'to-list ,bindings)))
(mapcar (lambda (,variable)
(second (find ,variable ,values :key 'first)))
',variables)))))
This version creates a lambda function where the arguments are the specified variables and the body is the provided body (so the declarations in the body are in an appropriate place), and then applies it to a list of values extracted from the result of the computed bindings.
Using LAMBDA or DESTRUCTURING-BIND
since I'm doing some "destructuring" of the arguments (in a bit a different way), I know which arguments must be present or have which
default values in case of missing optional and key arguments. So in
the first step I get a list of values and a flag whether an optional
or key argument was present or defaulted. In the second step I would
like to bind those values and/or present/default flag to local
variables to do some work with them
This is actually starting to sound like you can do what you need to by using a lambda function or destructuring-bind with keyword arguments. First, note that you can use any symbol as a keyword argument indicator. E.g.:
(apply (lambda (&key
((b bee) 'default-bee b?)
((c see) 'default-see c?))
(list bee b? see c?))
'(b 42))
;;=> (42 T DEFAULT-SEE NIL)
(destructuring-bind (&key ((b bee) 'default-bee b?)
((c see) 'default-see c?))
'(b 42)
(list bee b? see c?))
;;=> (42 T DEFAULT-SEE NIL)
So, if you just make your function return bindings as a list of keyword arguments, then in the destructuring or function application you can automatically bind corresponding variables, assign default values, and check whether non-default values were provided.
Acting a bit indirectly:
a solution that works for combining optional with key arguments or
rest/body with key arguments
Have you considered the not-entirely-uncommon paradigm of using a sub-list for the keywords?
e.g.
(defmacro something (&key (first 1) second) &body body) ... )
or, a practical use from Alexandria:
(defmacro with-output-to-file ((stream-name file-name
&rest args
&key (direction nil direction-p)
&allow-other-keys)
&body body)
So I thought one of the advantages of lisp (among other languages) is its ability to implement function factories (accept functions as arguments; return new functions). I want to use this capability to make small changes to a function and save it as a new function so that if changes are made to the original function, they are also reflected in the new function on which it is based. Note: I am not the one writing the original function so I can't necessarily encapsulate the common parts in a separate function to be called by both, which would be the obvious answer otherwise.
Toy example in emacs lisp (may not be the most ideal as it is a lisp-2):
I have a function, foo that is provided to me:
(defun foo (x y)
(+ x y)))
I want my new function to include a statement that allows me to change the value of a variable if a certain condition is met. For instance:
(defun newfoo (x y)
(if (condition-met-p x)
(setq x (transform x)))
(+ x y))
Please disregard that I could use defadvice in this particular example as I am more interested in the general task of modifying functions where defadvice may not apply. I believe I can modify the body with this form:
(setq conditional-transformation
'(if (condition-met x) (setq x (transform x))))
(setq newbody (append conditional-transformation
(nth 2 (symbol-function 'foo)))))
My questions are specifically how to
create a copy of foo to newfoo
and replace the body with the value
of newbody defined above. (I've
looked into fset, setf, and
function but perhaps not using
them properly.)
possibly wrap this in a function
called makenewfoo() or something
like this so I can invoke
makenewfoo(foo) and allow this to
create newfoo().
And, more generally,
is something like this is commonly
done or there is a more idiomatic
way to modify functions?
this is a very simple case, but is
there a more general way than
specifying the list element number
to nth for the modification. For
instance, the actual function is
more complex so is there a way to
recursively search down this
s-expression tree and test for a
particular syntax and insert this
conditional-transformation
expression before or after it
(possibly using equal), so it is
less sensitive to changes made in
the original function?
It does work in Emacs Lisp:
elisp> (defun foo (x y)
(+ x y))
foo
elisp> (fset 'newfoo
(append (lambda (x y)
(when (< x 2)
(setq x (* x 2))))
(cddr (symbol-function 'foo))))
(lambda
(x y)
(when
(< x 2)
(setq x
(* x 2)))
(+ x y))
elisp> (newfoo 1 3)
5
elisp> (newfoo 3 3)
6
But I really don't think that it is commonly done or idiomatic. You should use defadvice if you want to modify the behavior of functions.
As far as CL is concerned: Some implementations provide similar functions/macros (for example in CCL: ccl:advise), and you can specify :before, :after, and :around methods for generic functions.
Example code for insertion of expressions:
(defun find-node (elt tree)
(cond ((null tree) nil)
((equal (car tree) elt) tree)
((consp (car tree)) (let ((node (find-node elt (car tree))))
(if node node (find-node elt (cdr tree)))))
(t (find-node elt (cdr tree)))))
(defun insert-before (node elt)
(setcdr node (cons (car node) (cdr node)))
(setcar node elt))
(let* ((function (copy-tree (symbol-function 'foo)))
(node (find-node '(+ x y) function)))
(when node
(insert-before node '(if (< x 2) (setq x (* x 2))))
(fset 'newfoo function)))
I'm a Lisp beginner. I'm trying to memoize a recursive function for calculating the number of terms in a Collatz sequence (for problem 14 in Project Euler). My code as of yet is:
(defun collatz-steps (n)
(if (= 1 n) 0
(if (evenp n)
(1+ (collatz-steps (/ n 2)))
(1+ (collatz-steps (1+ (* 3 n)))))))
(defun p14 ()
(defvar m-collatz-steps (memoize #'collatz-steps))
(let
((maxsteps (funcall m-collatz-steps 2))
(n 2)
(steps))
(loop for i from 1 to 1000000
do
(setq steps (funcall m-collatz-steps i))
(cond
((> steps maxsteps)
(setq maxsteps steps)
(setq n i))
(t ())))
n))
(defun memoize (fn)
(let ((cache (make-hash-table :test #'equal)))
#'(lambda (&rest args)
(multiple-value-bind
(result exists)
(gethash args cache)
(if exists
result
(setf (gethash args cache)
(apply fn args)))))))
The memoize function is the same as the one given in the On Lisp book.
This code doesn't actually give any speedup compared to the non-memoized version. I believe it's due to the recursive calls calling the non-memoized version of the function, which sort of defeats the purpose. In that case, what is the correct way to do the memoization here? Is there any way to have all calls to the original function call the memoized version itself, removing the need for the special m-collatz-steps symbol?
EDIT: Corrected the code to have
(defvar m-collatz-steps (memoize #'collatz-steps))
which is what I had in my code.
Before the edit I had erroneously put:
(defvar collatz-steps (memoize #'collatz-steps))
Seeing that error gave me another idea, and I tried using this last defvar itself and changing the recursive calls to
(1+ (funcall collatz-steps (/ n 2)))
(1+ (funcall collatz-steps (1+ (* 3 n))))
This does seem to perform the memoization (speedup from about 60 seconds to 1.5 seconds), but requires changing the original function. Is there a cleaner solution which doesn't involve changing the original function?
I assume you're using Common-Lisp, which has separate namespaces for variable and function names. In order to memoize the function named by a symbol, you need to change its function binding, through the accessor `fdefinition':
(setf (fdefinition 'collatz-steps) (memoize #'collatz-steps))
(defun p14 ()
(let ((mx 0) (my 0))
(loop for x from 1 to 1000000
for y = (collatz-steps x)
when (< my y) do (setf my y mx x))
mx))
Here is a memoize function that rebinds the symbol function:
(defun memoize-function (function-name)
(setf (symbol-function function-name)
(let ((cache (make-hash-table :test #'equal)))
#'(lambda (&rest args)
(multiple-value-bind
(result exists)
(gethash args cache)
(if exists
result
(setf (gethash args cache)
(apply fn args)))))))
You would then do something like this:
(defun collatz-steps (n)
(if (= 1 n) 0
(if (evenp n)
(1+ (collatz-steps (/ n 2)))
(1+ (collatz-steps (1+ (* 3 n)))))))
(memoize-function 'collatz-steps)
I'll leave it up to you to make an unmemoize-function.
something like this:
(setf collatz-steps (memoize lambda (n)
(if (= 1 n) 0
(if (evenp n)
(1+ (collatz-steps (/ n 2)))
(1+ (collatz-steps (1+ (* 3 n))))))))
IOW: your original (non-memoized) function is anonymous, and you only give a name to the result of memoizing it.
Note a few things:
(defun foo (bar)
... (foo 3) ...)
Above is a function that has a call to itself.
In Common Lisp the file compiler can assume that FOO does not change. It will NOT call an updated FOO later. If you change the function binding of FOO, then the call of the original function will still go to the old function.
So memoizing a self recursive function will NOT work in the general case. Especially not if you are using a good compiler.
You can work around it to go always through the symbol for example: (funcall 'foo 3)
(DEFVAR ...) is a top-level form. Don't use it inside functions. If you have declared a variable, set it with SETQ or SETF later.
For your problem, I'd just use a hash table to store the intermediate results.
Changing the "original" function is necessary, because, as you say, there's no other way for the recursive call(s) to be updated to call the memoized version.
Fortunately, the way lisp works is to find the function by name each time it needs to be called. This means that it is sufficient to replace the function binding with the memoized version of the function, so that recursive calls will automatically look up and reenter through the memoization.
huaiyuan's code shows the key step:
(setf (fdefinition 'collatz-steps) (memoize #'collatz-steps))
This trick also works in Perl. In a language like C, however, a memoized version of a function must be coded separately.
Some lisp implementations provide a system called "advice", which provides a standardized structure for replacing functions with enhanced versions of themselves. In addition to functional upgrades like memoization, this can be extremely useful in debugging by inserting debug prints (or completely stopping and giving a continuable prompt) without modifying the original code.
This function is exactly the one Peter Norvig gives as an example of a function that seems like a good candidate for memoization, but which is not.
See figure 3 (the function 'Hailstone') of his original paper on memoization ("Using Automatic Memoization as a Software Engineering Tool in Real-World AI Systems").
So I'm guessing, even if you get the mechanics of memoization working, it won't really speed it up in this case.
A while ago I wrote a little memoization routine for Scheme that used a chain of closures to keep track of the memoized state:
(define (memoize op)
(letrec ((get (lambda (key) (list #f)))
(set (lambda (key item)
(let ((old-get get))
(set! get (lambda (new-key)
(if (equal? key new-key) (cons #t item)
(old-get new-key))))))))
(lambda args
(let ((ans (get args)))
(if (car ans) (cdr ans)
(let ((new-ans (apply op args)))
(set args new-ans)
new-ans))))))
This needs to be used like so:
(define fib (memoize (lambda (x)
(if (< x 2) x
(+ (fib (- x 1)) (fib (- x 2)))))))
I'm sure that this can be ported to your favorite lexically scoped Lisp flavor with ease.
I'd probably do something like:
(let ((memo (make-hash-table :test #'equal)))
(defun collatz-steps (n)
(or (gethash n memo)
(setf (gethash n memo)
(cond ((= n 1) 0)
((oddp n) (1+ (collatz-steps (+ 1 n n n))))
(t (1+ (collatz-steps (/ n 2)))))))))
It's not Nice and Functional, but, then, it's not much hassle and it does work. Downside is that you don't get a handy unmemoized version to test with and clearing the cache is bordering on "very difficult".