Google Common Lisp Style Guide say Avoid modifying local variables, try rebinding instead
What does it mean? What does rebinding mean in that sentence?
It means that you should create new variables instead of changing the value of old ones. For example, let's take the following code:
(defun foo (x)
(when (minusp x)
(setq x (- x)))
do something with x)
Instead, one should create a new binding and use that one instead:
(defun foo (x)
(let ((xabs (if (minusp x)
(- x)
x)))
do something with xabs)
The reason for this is that you will always know what a variable contains, since it will never change. If you want the new value, simply use the variable that holds that new value.
Now you might ask why this is so important? Well, some people have a stronger preference for this than others. Especially people who prefer to emphasise the functional aspect of Lisp will advocate this style. However, regardless of preference, it can be very useful to always be able to rely on the fact that variables doesn't change. Here's an example where this can be important:
(defun foo (x)
(let ((function #'(lambda () (format t "the value of x is ~a~%" x))))
(when (minusp x)
(setq x (- x)))
(other-function x)
function))
Then, the return value of FOO is a function that when called with print the value of x. But, the value will be that of x later in the function, the absolute value. This can be very surprising if the function is large and complicated.
I don't know Common Lisp well enough to answer with how to do this in Common Lisp, so I'm using Scheme for my example below. Suppose you're writing a function to return the factorial of a number. Here's a "modify local variables" approach to that function (you'll have to define your own while macro, but it's not hard):
(define (factorial n)
(define result 1)
(while (> n 0)
(set! result (* result n))
(set! n (- n 1)))
result)
Here's a "rebind local variables" approach to that function:
(define (factorial n)
(let loop ((n n)
(result 1))
(if (zero? n)
result
(loop (- n 1) (* result n)))))
In this case, loop is called with new values to rebind with each time. The same function can be written using the do macro (which, by the way, also uses rebinding, not modifying, at least in Scheme):
(define (factorial n)
(do ((n n (- n 1))
(result 1 (* result n)))
((zero? n) result)))
Related
Could someone explain to me what's going on in this very simple code snippet?
(defun test-a ()
(let ((x '(nil)))
(setcar x (cons 1 (car x)))
x))
Upon a calling (test-a) for the first time, I get the expected result: ((1)).
But to my surprise, calling it once more, I get ((1 1)), ((1 1 1)) and so on.
Why is this happening? Am I wrong to expect (test-a) to always return ((1))?
Also note that after re-evaluating the definition of test-a, the return result resets.
Also consider that this function works as I expect:
(defun test-b ()
(let ((x '(nil)))
(setq x (cons (cons 1 (car x))
(cdr x)))))
(test-b) always returns ((1)).
Why aren't test-a and test-b equivalent?
The Bad
test-a is self-modifying code. This is extremely dangerous. While the variable x disappears at the end of the let form, its initial value persists in the function object, and that is the value you are modifying. Remember that in Lisp a function is a first class object, which can be passed around (just like a number or a list), and, sometimes, modified. This is exactly what you are doing here: the initial value for x is a part of the function object and you are modifying it.
Let us actually see what is happening:
(symbol-function 'test-a)
=> (lambda nil (let ((x (quote (nil)))) (setcar x (cons 1 (car x))) x))
(test-a)
=> ((1))
(symbol-function 'test-a)
=> (lambda nil (let ((x (quote ((1))))) (setcar x (cons 1 (car x))) x))
(test-a)
=> ((1 1))
(symbol-function 'test-a)
=> (lambda nil (let ((x (quote ((1 1))))) (setcar x (cons 1 (car x))) x))
(test-a)
=> ((1 1 1))
(symbol-function 'test-a)
=> (lambda nil (let ((x (quote ((1 1 1))))) (setcar x (cons 1 (car x))) x))
The Good
test-b returns a fresh cons cell and thus is safe. The initial value of x is never modified. The difference between (setcar x ...) and (setq x ...) is that the former modifies the object already stored in the variable x while the latter stores a new object in x. The difference is similar to x.setField(42) vs. x = new MyObject(42) in C++.
The Bottom Line
In general, it is best to treat quoted data like '(1) as constants - do not modify them:
quote returns the argument, without evaluating it. (quote x) yields x.
Warning: quote does not construct its return value, but just returns
the value that was pre-constructed by the Lisp reader (see info node
Printed Representation). This means that (a . b) is not
identical to (cons 'a 'b): the former does not cons. Quoting should
be reserved for constants that will never be modified by side-effects,
unless you like self-modifying code. See the common pitfall in info
node Rearrangement for an example of unexpected results when
a quoted object is modified.
If you need to modify a list, create it with list or cons or copy-list instead of quote.
See more examples.
PS1. This has been duplicated on Emacs.
PS2. See also Why does this function return a different value every time? for an identical Common Lisp issue.
PS3. See also Issue CONSTANT-MODIFICATION.
I found the culprit is indeed 'quote. Here's its doc-string:
Return the argument, without evaluating it.
...
Warning: `quote' does not construct its return value, but just returns
the value that was pre-constructed by the Lisp reader
...
Quoting should be reserved for constants that will
never be modified by side-effects, unless you like self-modifying code.
I also rewrote for convenience
(setq test-a
(lambda () ((lambda (x) (setcar x (cons 1 (car x))) x) (quote (nil)))))
and then used
(funcall test-a)
to see how 'test-a was changing.
It looks like the '(nil) in your (let) is only evaluated once. When you (setcar), each call is modifying the same list in-place. You can make (test-a) work if you replace the '(nil) with (list (list)), although I presume there's a more elegant way to do it.
(test-b) constructs a totally new list from cons cells each time, which is why it works differently.
So I thought one of the advantages of lisp (among other languages) is its ability to implement function factories (accept functions as arguments; return new functions). I want to use this capability to make small changes to a function and save it as a new function so that if changes are made to the original function, they are also reflected in the new function on which it is based. Note: I am not the one writing the original function so I can't necessarily encapsulate the common parts in a separate function to be called by both, which would be the obvious answer otherwise.
Toy example in emacs lisp (may not be the most ideal as it is a lisp-2):
I have a function, foo that is provided to me:
(defun foo (x y)
(+ x y)))
I want my new function to include a statement that allows me to change the value of a variable if a certain condition is met. For instance:
(defun newfoo (x y)
(if (condition-met-p x)
(setq x (transform x)))
(+ x y))
Please disregard that I could use defadvice in this particular example as I am more interested in the general task of modifying functions where defadvice may not apply. I believe I can modify the body with this form:
(setq conditional-transformation
'(if (condition-met x) (setq x (transform x))))
(setq newbody (append conditional-transformation
(nth 2 (symbol-function 'foo)))))
My questions are specifically how to
create a copy of foo to newfoo
and replace the body with the value
of newbody defined above. (I've
looked into fset, setf, and
function but perhaps not using
them properly.)
possibly wrap this in a function
called makenewfoo() or something
like this so I can invoke
makenewfoo(foo) and allow this to
create newfoo().
And, more generally,
is something like this is commonly
done or there is a more idiomatic
way to modify functions?
this is a very simple case, but is
there a more general way than
specifying the list element number
to nth for the modification. For
instance, the actual function is
more complex so is there a way to
recursively search down this
s-expression tree and test for a
particular syntax and insert this
conditional-transformation
expression before or after it
(possibly using equal), so it is
less sensitive to changes made in
the original function?
It does work in Emacs Lisp:
elisp> (defun foo (x y)
(+ x y))
foo
elisp> (fset 'newfoo
(append (lambda (x y)
(when (< x 2)
(setq x (* x 2))))
(cddr (symbol-function 'foo))))
(lambda
(x y)
(when
(< x 2)
(setq x
(* x 2)))
(+ x y))
elisp> (newfoo 1 3)
5
elisp> (newfoo 3 3)
6
But I really don't think that it is commonly done or idiomatic. You should use defadvice if you want to modify the behavior of functions.
As far as CL is concerned: Some implementations provide similar functions/macros (for example in CCL: ccl:advise), and you can specify :before, :after, and :around methods for generic functions.
Example code for insertion of expressions:
(defun find-node (elt tree)
(cond ((null tree) nil)
((equal (car tree) elt) tree)
((consp (car tree)) (let ((node (find-node elt (car tree))))
(if node node (find-node elt (cdr tree)))))
(t (find-node elt (cdr tree)))))
(defun insert-before (node elt)
(setcdr node (cons (car node) (cdr node)))
(setcar node elt))
(let* ((function (copy-tree (symbol-function 'foo)))
(node (find-node '(+ x y) function)))
(when node
(insert-before node '(if (< x 2) (setq x (* x 2))))
(fset 'newfoo function)))
I'm currently working on a LISP exercise for a small project and need severe help. This may be more or less of a beginner's question but I'm absolutely lost on writing a certain function that takes in two unevaluated functions and spits out the result dependent on if the variables were given an assignment or not.
An example would be
(setq p1 '(+ x (* x (- y (/ z 2)))))
Where
(evalexp p1 '( (x 2) (z 8) ))
returns (+ 2 (* 2 (- y 4)))
My goal is to write the evalexp function but I can't even think of where to start.
So far I have
(defun evalexp (e b) )
.. not very much. If anyone could please help or lead me in a good direction I'd be more than appreciative.
Here's a full solution. It's pretty straightforward, so I'll leave out a full explanation. Ask me in the comments if there's anything you can't figure out yourself.
(Using eval to do the actual evaluation might not be what you want in your exercise/project. Look up "meta-circular interpreter" for another way.)
(defun apply-env (exp env)
(reduce (lambda (exp bdg) (subst (cadr bdg) (car bdg) exp))
env :initial-value exp))
(defun try-eval (exp)
(if (atom exp)
exp
(let ((exp (mapcar #'try-eval exp)))
(if (every #'numberp (cdr exp))
(eval exp)
exp))))
(defun evalexp (exp env)
(try-eval (apply-env exp env)))
Here's a hint, this is how you might do it (in pseudocode):
function replace(vars, list):
for each element of list:
if it's an atom:
if there's an association in vars:
replace atom with value in vars
else:
leave atom alone
else:
recursively apply replace to the sublist
There will certainly be some details to work out as you convert this to Lisp code.
The following procedure is valid in both scheme r6rs and Racket:
;; create a list of all the numbers from 1 to n
(define (make-nums n)
(do [(x n (- x 1)) (lst (list) (cons x lst))]
((= x 0)
lst)))
I've tested it for both r6rs and Racket and it does work properly, but I only know that for sure for DrRacket.
My question is if it is guaranteed that the step expressions ((- x 1) and (cons x lst) in this case) will be evaluated in order. If it's not guaranteed, then my procedure isn't very stable.
I didn't see anything specifying this in the standards for either language, but I'm asking here because when I tested it was evaulated in order.
They're generally not guaranteed to be evaluated in order, but the result will still be the same. This is because there are no side-effects here -- the loop doesn't change x or lst, it just rebinds them to new values, so the order in which the two step expressions are evaluated is irrelevant.
To see this, start with a cleaner-looking version of your code:
(define (make-nums n)
(do ([x n (- x 1)] [lst null (cons x lst)])
[(zero? x) lst]))
translate to a named-let:
(define (make-nums n)
(let loop ([x n] [lst null])
(if (zero? x)
lst
(loop (- x 1) (cons x lst)))))
and further translate that to a helper function (which is what a named-let really is):
(define (make-nums n)
(define (loop x lst)
(if (zero? x)
lst
(loop (- x 1) (cons x lst))))
(loop n null))
It should be clear now that the order of evaluating the two expressions in the recursive loop call doesn't make it do anything different.
Finally, note that in Racket evaluation is guaranteed to be left-to-right. This matters when there are side-effects -- Racket prefers a predictable behavior, whereas others object to it, claiming that this leads people to code that implicitly relies on this. A common small example that shows the difference is:
(list (read-line) (read-line))
which in Racket is guaranteed to return a list of the first line read, and then the second. Other implementations might return the two lines in a different order.
I'm a Lisp beginner. I'm trying to memoize a recursive function for calculating the number of terms in a Collatz sequence (for problem 14 in Project Euler). My code as of yet is:
(defun collatz-steps (n)
(if (= 1 n) 0
(if (evenp n)
(1+ (collatz-steps (/ n 2)))
(1+ (collatz-steps (1+ (* 3 n)))))))
(defun p14 ()
(defvar m-collatz-steps (memoize #'collatz-steps))
(let
((maxsteps (funcall m-collatz-steps 2))
(n 2)
(steps))
(loop for i from 1 to 1000000
do
(setq steps (funcall m-collatz-steps i))
(cond
((> steps maxsteps)
(setq maxsteps steps)
(setq n i))
(t ())))
n))
(defun memoize (fn)
(let ((cache (make-hash-table :test #'equal)))
#'(lambda (&rest args)
(multiple-value-bind
(result exists)
(gethash args cache)
(if exists
result
(setf (gethash args cache)
(apply fn args)))))))
The memoize function is the same as the one given in the On Lisp book.
This code doesn't actually give any speedup compared to the non-memoized version. I believe it's due to the recursive calls calling the non-memoized version of the function, which sort of defeats the purpose. In that case, what is the correct way to do the memoization here? Is there any way to have all calls to the original function call the memoized version itself, removing the need for the special m-collatz-steps symbol?
EDIT: Corrected the code to have
(defvar m-collatz-steps (memoize #'collatz-steps))
which is what I had in my code.
Before the edit I had erroneously put:
(defvar collatz-steps (memoize #'collatz-steps))
Seeing that error gave me another idea, and I tried using this last defvar itself and changing the recursive calls to
(1+ (funcall collatz-steps (/ n 2)))
(1+ (funcall collatz-steps (1+ (* 3 n))))
This does seem to perform the memoization (speedup from about 60 seconds to 1.5 seconds), but requires changing the original function. Is there a cleaner solution which doesn't involve changing the original function?
I assume you're using Common-Lisp, which has separate namespaces for variable and function names. In order to memoize the function named by a symbol, you need to change its function binding, through the accessor `fdefinition':
(setf (fdefinition 'collatz-steps) (memoize #'collatz-steps))
(defun p14 ()
(let ((mx 0) (my 0))
(loop for x from 1 to 1000000
for y = (collatz-steps x)
when (< my y) do (setf my y mx x))
mx))
Here is a memoize function that rebinds the symbol function:
(defun memoize-function (function-name)
(setf (symbol-function function-name)
(let ((cache (make-hash-table :test #'equal)))
#'(lambda (&rest args)
(multiple-value-bind
(result exists)
(gethash args cache)
(if exists
result
(setf (gethash args cache)
(apply fn args)))))))
You would then do something like this:
(defun collatz-steps (n)
(if (= 1 n) 0
(if (evenp n)
(1+ (collatz-steps (/ n 2)))
(1+ (collatz-steps (1+ (* 3 n)))))))
(memoize-function 'collatz-steps)
I'll leave it up to you to make an unmemoize-function.
something like this:
(setf collatz-steps (memoize lambda (n)
(if (= 1 n) 0
(if (evenp n)
(1+ (collatz-steps (/ n 2)))
(1+ (collatz-steps (1+ (* 3 n))))))))
IOW: your original (non-memoized) function is anonymous, and you only give a name to the result of memoizing it.
Note a few things:
(defun foo (bar)
... (foo 3) ...)
Above is a function that has a call to itself.
In Common Lisp the file compiler can assume that FOO does not change. It will NOT call an updated FOO later. If you change the function binding of FOO, then the call of the original function will still go to the old function.
So memoizing a self recursive function will NOT work in the general case. Especially not if you are using a good compiler.
You can work around it to go always through the symbol for example: (funcall 'foo 3)
(DEFVAR ...) is a top-level form. Don't use it inside functions. If you have declared a variable, set it with SETQ or SETF later.
For your problem, I'd just use a hash table to store the intermediate results.
Changing the "original" function is necessary, because, as you say, there's no other way for the recursive call(s) to be updated to call the memoized version.
Fortunately, the way lisp works is to find the function by name each time it needs to be called. This means that it is sufficient to replace the function binding with the memoized version of the function, so that recursive calls will automatically look up and reenter through the memoization.
huaiyuan's code shows the key step:
(setf (fdefinition 'collatz-steps) (memoize #'collatz-steps))
This trick also works in Perl. In a language like C, however, a memoized version of a function must be coded separately.
Some lisp implementations provide a system called "advice", which provides a standardized structure for replacing functions with enhanced versions of themselves. In addition to functional upgrades like memoization, this can be extremely useful in debugging by inserting debug prints (or completely stopping and giving a continuable prompt) without modifying the original code.
This function is exactly the one Peter Norvig gives as an example of a function that seems like a good candidate for memoization, but which is not.
See figure 3 (the function 'Hailstone') of his original paper on memoization ("Using Automatic Memoization as a Software Engineering Tool in Real-World AI Systems").
So I'm guessing, even if you get the mechanics of memoization working, it won't really speed it up in this case.
A while ago I wrote a little memoization routine for Scheme that used a chain of closures to keep track of the memoized state:
(define (memoize op)
(letrec ((get (lambda (key) (list #f)))
(set (lambda (key item)
(let ((old-get get))
(set! get (lambda (new-key)
(if (equal? key new-key) (cons #t item)
(old-get new-key))))))))
(lambda args
(let ((ans (get args)))
(if (car ans) (cdr ans)
(let ((new-ans (apply op args)))
(set args new-ans)
new-ans))))))
This needs to be used like so:
(define fib (memoize (lambda (x)
(if (< x 2) x
(+ (fib (- x 1)) (fib (- x 2)))))))
I'm sure that this can be ported to your favorite lexically scoped Lisp flavor with ease.
I'd probably do something like:
(let ((memo (make-hash-table :test #'equal)))
(defun collatz-steps (n)
(or (gethash n memo)
(setf (gethash n memo)
(cond ((= n 1) 0)
((oddp n) (1+ (collatz-steps (+ 1 n n n))))
(t (1+ (collatz-steps (/ n 2)))))))))
It's not Nice and Functional, but, then, it's not much hassle and it does work. Downside is that you don't get a handy unmemoized version to test with and clearing the cache is bordering on "very difficult".