Currently, i have been reading the book the scheme programming language written by Kent Dybvig.
In section 5.7, it implements memorized lazy eval in scheme using the scheme macro system.
The source code as
(define-syntax delay
(syntax-rules ()
[(_ expr) (make-promise (lambda () expr))]))
(define make-promise
(lambda (p)
(let ([val #f] [set? #f])
(lambda ()
(unless set?
(let ([x (p)])
(unless set?
(set! val x)
(set! set? #t))))
val))))
(define force
(lambda (promise)
(promise)))
But i can not understand why the variable set? need to be tested twice in the procedure make-promise.
This is the reason from the book
The second test of the variable set? in make-promise is necessary in the event that, as a result of
applying p, the promise is recursively forced. Since a promise must always return the same value, the result of
the first application of p to complete is returned.
which i can not understand
Could anyone explain it to me? Thanks!
The key is that force may reenter itself. Maybe an example can help you understand this:
(define x 5)
(letrec ([f (delay
(if (zero? x)
0
(begin
(set! x (- x 1))
(+ (force f) 1))))])
(force f))
The result will be 0, because the inner force call returns 0.
If without the second test, the result will be 5. In this situation, every (force f) returns different values.
Related
Prelude
In Raku there's a notion called infinite list AKA lazy list which is defined and used like:
my #inf = (1,2,3 ... Inf);
for #inf { say $_;
exit if $_ == 7 }
# => OUTPUT
1
2
3
4
5
6
7
I'd like to implement this sort of thing in Common Lisp, specifically an infinite list of consecutive integers like:
(defun inf (n)
("the implementation"))
such that
(inf 5)
=> (5 6 7 8 9 10 .... infinity)
;; hypothetical output just for the demo purposes. It won't be used in reality
Then I'll use it for lazy evaluation like this:
(defun try () ;; catch and dolist
(catch 'foo ;; are just for demo purposes
(dolist (n (inf 1) 'done)
(format t "~A~%" n)
(when (= n 7)
(throw 'foo x)))))
CL-USER> (try)
1
2
3
4
5
6
7
; Evaluation aborted.
How can I implement such an infinite list in CL in the most practical way?
A good pedagogical approach to this is to define things which are sometimes called 'streams'. The single best introduction to doing this that I know of is in Structure and Interpretation of Computer Programs. Streams are introduced in section 3.5, but don't just read that: read the book, seriously: it is a book everyone interested in programming should read.
SICP uses Scheme, and this sort of thing is more natural in Scheme. But it can be done in CL reasonably easily. What I've written below is rather 'Schemy' CL: in particular I just assume tail calls are optimised. That's not a safe assumption in CL, but it's good enough to see how you can build these concepts into a language which does not already have them, if your language is competent.
First of all we need a construct which supports lazy evaluation: we need to be able to 'delay' something to create a 'promise' which will be evaluated only when it needs to be. Well, what functions do is evaluate their body only when they are asked to, so we'll use them:
(defmacro delay (form)
(let ((stashn (make-symbol "STASH"))
(forcedn (make-symbol "FORCED")))
`(let ((,stashn nil)
(,forcedn nil))
(lambda ()
(if ,forcedn
,stashn
(setf ,forcedn t
,stashn ,form))))))
(defun force (thing)
(funcall thing))
delay is mildly fiddly, it wants to make sure that a promise is forced only once, and it also wants to make sure that the form being delayed doesn't get infected by the state it uses to do that. You can trace the expansion of delay to see what it makes:
(delay (print 1))
-> (let ((#:stash nil) (#:forced nil))
(lambda ()
(if #:forced #:stash (setf #:forced t #:stash (print 1)))))
This is fine.
So now, we'll invent streams: streams are like conses (they are conses!) but their cdrs are delayed:
(defmacro cons-stream (car cdr)
`(cons ,car (delay ,cdr)))
(defun stream-car (s)
(car s))
(defun stream-cdr (s)
(force (cdr s)))
OK, let's write a function to get the nth element of a stream:
(defun stream-nth (n s)
(cond ((null s)
nil)
((= n 0) (stream-car s))
(t
(stream-nth (1- n) (stream-cdr s)))))
And we can test this:
> (stream-nth 2
(cons-stream 0 (cons-stream 1 (cons-stream 2 nil))))
2
And now we can write a function to enumerate an interval in the naturals, which by default will be an half-infinite interval:
(defun stream-enumerate-interval (low &optional (high nil))
(if (and high (> low high))
nil
(cons-stream
low
(stream-enumerate-interval (1+ low) high))))
And now:
> (stream-nth 1000 (stream-enumerate-interval 0))
1000
And so on.
Well, we'd like some kind of macro which lets us traverse a stream: something like dolist, but for streams. Well we can do this by first writing a function which will call a function for each element in the stream (this is not the way I'd do this in production CL code, but it's fine here):
(defun call/stream-elements (f s)
;; Call f on the elements of s, returning NIL
(if (null s)
nil
(progn
(funcall f (stream-car s))
(call/stream-elements f (stream-cdr s)))))
And now
(defmacro do-stream ((e s &optional (r 'nil)) &body forms)
`(progn
(call/stream-elements (lambda (,e)
,#forms)
,s)
,r))
And now, for instance
(defun look-for (v s)
;; look for an element of S which is EQL to V
(do-stream (e s (values nil nil))
(when (eql e v)
(return-from look-for (values e t)))))
And we can then say
> (look-for 100 (stream-enumerate-interval 0))
100
t
Well, there is a lot more mechanism you need to make streams really useful: you need to be able to combine them, append them and so on. SICP has many of these functions, and they're generally easy to turn into CL, but too long here.
For practical purposes it would be wise to use existing libraries, but since the question is about how to implemented lazy lists, we will do it from scratch.
Closures
Lazy iteration is a matter of producing an object that can generate the new value of a lazy sequence each time it is asked to do so.
A simple approach for this is to return a closure, i.e. a function that closes over variables, which produces values while updating its state by side-effect.
If you evaluate:
(let ((a 0))
(lambda () (incf a)))
You obtain a function object that has a local state, namely here the variable named a.
This is a lexical binding to a location that is exclusive to this function, if you evaluate a second time the same expression, you'll obtain a different anonymous function that has its own local state.
When you call the closure, the value stored in a in incremented and its value is returned.
Let's bind this closure to a variable named counter, call it multiple times and store the successive results in a list:
(let ((counter (let ((a 0))
(lambda () (incf a)))))
(list (funcall counter)
(funcall counter)
(funcall counter)
(funcall counter)))
The resulting list is:
(1 2 3 4)
Simple iterator
In your case, you want to have an iterator that starts counting from 5 when writing:
(inf 5)
This can implemented as follows:
(defun inf (n)
(lambda ()
(shiftf n (1+ n))))
Here is there is no need to add a let, the lexical binding of an argument to n is done when calling the function.
We assign n to a different value within the body over time.
More precisely, SHIFTF assigns n to (1+ n), but returns the previous value of n.
For example:
(let ((it (inf 5)))
(list (funcall it)
(funcall it)
(funcall it)
(funcall it)))
Which gives:
(5 6 7 8)
Generic iterator
The standard dolist expects a proper list as an input, there is no way you can put another kind of data and expect it to work (or maybe in an implementation-specific way).
We need a similar macro to iterate over all the values in an arbitrary iterator.
We also need to specify when iteration stops.
There are multiple possibilities here, let's define a basic iteration protocol as follows:
we can call make-iterator on any object, along with arbitrary arguments, to obtain an iterator
we can call next on an iterator to obtain the next value.
More precisely, if there is a value, next returns the value and T as a secondary value; otherwise, next returns NIL.
Let's define two generic functions:
(defgeneric make-iterator (object &key)
(:documentation "create an iterator for OBJECT and arguments ARGS"))
(defgeneric next (iterator)
(:documentation "returns the next value and T as a secondary value, or NIL"))
Using generic functions allows the user to define custom iterators, as long as they respect the specified behaviour above.
Instead of using dolist, which only works with eager sequences, we define our own macro: for.
It hides calls to make-iterator and next from the user.
In other words, for takes an object and iterates over it.
We can skip iteration with (return v) since for is implemented with loop.
(defmacro for ((value object &rest args) &body body)
(let ((it (gensym)) (exists (gensym)))
`(let ((,it (make-iterator ,object ,#args)))
(loop
(multiple-value-bind (,value ,exists) (next ,it)
(unless ,exists
(return))
,#body)))))
We assume any function object can act as an iterator, so we specialize next for values f of class function, so that the function f gets called:
(defmethod next ((f function))
"A closure is an interator"
(funcall f))
Also, we can also specialize make-iterator to make closures their own iterators (I see no other good default behaviour to provide for closures):
(defmethod make-iterator ((function function) &key)
function)
Vector iterator
For example, we can built an iterator for vectors as follows. We specialize make-iterator for values (here named vec) of class vector.
The returned iterator is a closure, so we will be able to call next on it.
The method accepts a :start argument defaulting to zero:
(defmethod make-iterator ((vec vector) &key (start 0))
"Vector iterator"
(let ((index start))
(lambda ()
(when (array-in-bounds-p vec index)
(values (aref vec (shiftf index (1+ index))) t)))))
You can now write:
(for (v "abcdefg" :start 2)
(print v))
And this prints the following characters:
#\c
#\d
#\e
#\f
#\g
List iterator
Likewise, we can build a list iterator.
Here to demonstrate other kind of iterators, let's have a custom cursor type.
(defstruct list-cursor head)
The cursor is an object which keeps a reference to the current cons-cell in the list being visited, or NIL.
(defmethod make-iterator ((list list) &key)
"List iterator"
(make-list-cursor :head list))
And we define next as follows, specializeing on list-cursor:
(defmethod next ((cursor list-cursor))
(when (list-cursor-head cursor)
(values (pop (list-cursor-head cursor)) t)))
Ranges
Common Lisp also allows methods to be specialized with EQL specializers, which means the object we give to for might be a specific keyword, for example :range.
(defmethod make-iterator ((_ (eql :range)) &key (from 0) (to :infinity) (by 1))
(check-type from number)
(check-type to (or number (eql :infinity)))
(check-type by number)
(let ((counter from))
(case to
(:infinity
(lambda () (values (incf counter by) t)))
(t
(lambda ()
(when (< counter to)
(values (incf counter by) T)))))))
A possible call for make-iterator would be:
(make-iterator :range :from 0 :to 10 :by 2)
This also returns a closure.
Here, for example, you would iterate over a range as follows:
(for (v :range :from 0 :to 10 :by 2)
(print v))
The above expands as:
(let ((#:g1463 (make-iterator :range :from 0 :to 10 :by 2)))
(loop
(multiple-value-bind (v #:g1464)
(next #:g1463)
(unless #:g1464 (return))
(print v))))
Finally, if we add small modification to inf (adding secondary value):
(defun inf (n)
(lambda ()
(values (shiftf n (1+ n)) T)))
We can write:
(for (v (inf 5))
(print v)
(when (= v 7)
(return)))
Which prints:
5
6
7
I'll show it with a library:
How to create and consume an infinite list of integers with the GTWIWTG generators library
This library, called "Generators The Way I Want Them Generated", allows to do three things:
create generators (iterators)
combine them
consume them (once).
It is not unsimilar to the nearly-classic Series.
Install the lib with (ql:quickload "gtwiwtg"). I will work in its package: (in-package :gtwiwtg).
Create a generator for an infinite list of integers, start from 0:
GTWIWTG> (range)
#<RANGE-BACKED-GENERATOR! {10042B4D83}>
We can also specify its :from, :to, :by and :inclusive parameters.
Combine this generator with others: not needed here.
Iterate over it and stop:
GTWIWTG> (for x *
(print x)
(when (= x 7)
(return)))
0
1
2
3
4
5
6
7
T
This solution is very practical :)
What exactly is different between these implementations of 'when'?
(define-syntax when
(syntax-rules ()
((_ pred b1 ...)
(if pred (begin b1 ...)))))
vs.
(define (my-when pred b1 ...)
(if pred (begin b1 ...)))
For example, when 'my-when' is used in this for loop macro:
(define-syntax for
(syntax-rules ()
((_ (i from to) b1 ...)
(let loop((i from))
(my-when (< i to)
b1 ...
(loop (+ i 1)))))))
an error occurs:
(for (i 0 10) (display i))
; Aborting!: maximum recursion depth exceeded
I do not think 'when' can be implemented as a function, but I do not know why...
Scheme has strict semantics.
This means that all of a function's parameters are evaluated before the function is applied to them.
Macros take source code and produce source code - they don't evaluate any of their parameters.
(Or, well, I suppose they do, but their parameters are syntax - language elements - rather than what you normally think of as values, such as numbers or strings. Macro programming is meta-programming. It's important to be aware of which level you're programming at.)
In your example this means that when my-when is a function, (loop (+ i 1)) must be evaluated before my-when can be applied to it.
This leads to an infinite recursion.
When it's a macro, the my-when form is first replaced with the equivalent if-form
(if (< i to)
(begin
b1 ...
(loop (+ i 1))))
and then the whole thing is evaluated, which means that (loop (+ i 1)) only gets evaluated when the condition is true.
If you implement when as a procedure like you did, then all arguments are evaluated. In your for implementation, the evaluation would be processed like this:
evaluate (< i to)
evaluate expansion result of b1 ...
evaluate (loop (+ i 1)) <- here goes into infinite loop!
evaluate my-when
Item 1-3 can be reverse or undefined order depending on your implementation but the point is nr. 4. If my-when is implemented as a macro, then the macro is the first one to be evaluated.
If you really need to implement with a procedure, then you need to use sort of delaying trick such as thunk. For example:
(define (my-when pred body) (if (pred) (body)))
(my-when (lambda () (< i 10)) (lambda () (display i) (loop (+ i 1))))
Google Common Lisp Style Guide say Avoid modifying local variables, try rebinding instead
What does it mean? What does rebinding mean in that sentence?
It means that you should create new variables instead of changing the value of old ones. For example, let's take the following code:
(defun foo (x)
(when (minusp x)
(setq x (- x)))
do something with x)
Instead, one should create a new binding and use that one instead:
(defun foo (x)
(let ((xabs (if (minusp x)
(- x)
x)))
do something with xabs)
The reason for this is that you will always know what a variable contains, since it will never change. If you want the new value, simply use the variable that holds that new value.
Now you might ask why this is so important? Well, some people have a stronger preference for this than others. Especially people who prefer to emphasise the functional aspect of Lisp will advocate this style. However, regardless of preference, it can be very useful to always be able to rely on the fact that variables doesn't change. Here's an example where this can be important:
(defun foo (x)
(let ((function #'(lambda () (format t "the value of x is ~a~%" x))))
(when (minusp x)
(setq x (- x)))
(other-function x)
function))
Then, the return value of FOO is a function that when called with print the value of x. But, the value will be that of x later in the function, the absolute value. This can be very surprising if the function is large and complicated.
I don't know Common Lisp well enough to answer with how to do this in Common Lisp, so I'm using Scheme for my example below. Suppose you're writing a function to return the factorial of a number. Here's a "modify local variables" approach to that function (you'll have to define your own while macro, but it's not hard):
(define (factorial n)
(define result 1)
(while (> n 0)
(set! result (* result n))
(set! n (- n 1)))
result)
Here's a "rebind local variables" approach to that function:
(define (factorial n)
(let loop ((n n)
(result 1))
(if (zero? n)
result
(loop (- n 1) (* result n)))))
In this case, loop is called with new values to rebind with each time. The same function can be written using the do macro (which, by the way, also uses rebinding, not modifying, at least in Scheme):
(define (factorial n)
(do ((n n (- n 1))
(result 1 (* result n)))
((zero? n) result)))
In this page there is a comment after the post that gives a very short implementation of amb as a procedure:
(define (amb-backtrack)
(error "no solution found"))
(define (amb . args)
(call/cc (lambda (return)
(let ((backtrack amb-backtrack))
(map (lambda (x)
(call/cc (lambda (k)
(set! amb-backtrack k)
(return x))))
args)
(backtrack 'fail)))))
But I usually see amb implemented as a macro -- in the schemers.org FAQ, and also in Dorai Sitaram's book:
(define amb-fail '*)
(define initialize-amb-fail
(lambda ()
(set! amb-fail
(lambda ()
(error "amb tree exhausted")))))
(initialize-amb-fail)
(define-macro amb
(lambda alts...
`(let ((+prev-amb-fail amb-fail))
(call/cc
(lambda (+sk)
,#(map (lambda (alt)
`(call/cc
(lambda (+fk)
(set! amb-fail
(lambda ()
(set! amb-fail +prev-amb-fail)
(+fk 'fail)))
(+sk ,alt))))
alts...)
(+prev-amb-fail))))))
So -- the macro version is longer, and a little harder to understand. I could not see any advantages of it over the procedure version, and of course I would rather use a procedure than a macro. Is there anything I missed?
The difference is that a procedure call always evaluates all the arguments.
(amb 1 (very-expensive-computation))
will, with the procedure version of amb, perform the very-expensive-computation, then yield 1. If yielding 1 suffices for all further computation, then you've wasted a lot of time on a computation whose result value is never used. Even worse things happen, as #Eli Barzilay mentions in a comment, when amb is used to model an infinite non-determinism such as generating all natural numbers.
The macro version avoids this and its behavior is therefore closer to that of non-deterministic programming languages such as Prolog.
I'm a Lisp beginner. I'm trying to memoize a recursive function for calculating the number of terms in a Collatz sequence (for problem 14 in Project Euler). My code as of yet is:
(defun collatz-steps (n)
(if (= 1 n) 0
(if (evenp n)
(1+ (collatz-steps (/ n 2)))
(1+ (collatz-steps (1+ (* 3 n)))))))
(defun p14 ()
(defvar m-collatz-steps (memoize #'collatz-steps))
(let
((maxsteps (funcall m-collatz-steps 2))
(n 2)
(steps))
(loop for i from 1 to 1000000
do
(setq steps (funcall m-collatz-steps i))
(cond
((> steps maxsteps)
(setq maxsteps steps)
(setq n i))
(t ())))
n))
(defun memoize (fn)
(let ((cache (make-hash-table :test #'equal)))
#'(lambda (&rest args)
(multiple-value-bind
(result exists)
(gethash args cache)
(if exists
result
(setf (gethash args cache)
(apply fn args)))))))
The memoize function is the same as the one given in the On Lisp book.
This code doesn't actually give any speedup compared to the non-memoized version. I believe it's due to the recursive calls calling the non-memoized version of the function, which sort of defeats the purpose. In that case, what is the correct way to do the memoization here? Is there any way to have all calls to the original function call the memoized version itself, removing the need for the special m-collatz-steps symbol?
EDIT: Corrected the code to have
(defvar m-collatz-steps (memoize #'collatz-steps))
which is what I had in my code.
Before the edit I had erroneously put:
(defvar collatz-steps (memoize #'collatz-steps))
Seeing that error gave me another idea, and I tried using this last defvar itself and changing the recursive calls to
(1+ (funcall collatz-steps (/ n 2)))
(1+ (funcall collatz-steps (1+ (* 3 n))))
This does seem to perform the memoization (speedup from about 60 seconds to 1.5 seconds), but requires changing the original function. Is there a cleaner solution which doesn't involve changing the original function?
I assume you're using Common-Lisp, which has separate namespaces for variable and function names. In order to memoize the function named by a symbol, you need to change its function binding, through the accessor `fdefinition':
(setf (fdefinition 'collatz-steps) (memoize #'collatz-steps))
(defun p14 ()
(let ((mx 0) (my 0))
(loop for x from 1 to 1000000
for y = (collatz-steps x)
when (< my y) do (setf my y mx x))
mx))
Here is a memoize function that rebinds the symbol function:
(defun memoize-function (function-name)
(setf (symbol-function function-name)
(let ((cache (make-hash-table :test #'equal)))
#'(lambda (&rest args)
(multiple-value-bind
(result exists)
(gethash args cache)
(if exists
result
(setf (gethash args cache)
(apply fn args)))))))
You would then do something like this:
(defun collatz-steps (n)
(if (= 1 n) 0
(if (evenp n)
(1+ (collatz-steps (/ n 2)))
(1+ (collatz-steps (1+ (* 3 n)))))))
(memoize-function 'collatz-steps)
I'll leave it up to you to make an unmemoize-function.
something like this:
(setf collatz-steps (memoize lambda (n)
(if (= 1 n) 0
(if (evenp n)
(1+ (collatz-steps (/ n 2)))
(1+ (collatz-steps (1+ (* 3 n))))))))
IOW: your original (non-memoized) function is anonymous, and you only give a name to the result of memoizing it.
Note a few things:
(defun foo (bar)
... (foo 3) ...)
Above is a function that has a call to itself.
In Common Lisp the file compiler can assume that FOO does not change. It will NOT call an updated FOO later. If you change the function binding of FOO, then the call of the original function will still go to the old function.
So memoizing a self recursive function will NOT work in the general case. Especially not if you are using a good compiler.
You can work around it to go always through the symbol for example: (funcall 'foo 3)
(DEFVAR ...) is a top-level form. Don't use it inside functions. If you have declared a variable, set it with SETQ or SETF later.
For your problem, I'd just use a hash table to store the intermediate results.
Changing the "original" function is necessary, because, as you say, there's no other way for the recursive call(s) to be updated to call the memoized version.
Fortunately, the way lisp works is to find the function by name each time it needs to be called. This means that it is sufficient to replace the function binding with the memoized version of the function, so that recursive calls will automatically look up and reenter through the memoization.
huaiyuan's code shows the key step:
(setf (fdefinition 'collatz-steps) (memoize #'collatz-steps))
This trick also works in Perl. In a language like C, however, a memoized version of a function must be coded separately.
Some lisp implementations provide a system called "advice", which provides a standardized structure for replacing functions with enhanced versions of themselves. In addition to functional upgrades like memoization, this can be extremely useful in debugging by inserting debug prints (or completely stopping and giving a continuable prompt) without modifying the original code.
This function is exactly the one Peter Norvig gives as an example of a function that seems like a good candidate for memoization, but which is not.
See figure 3 (the function 'Hailstone') of his original paper on memoization ("Using Automatic Memoization as a Software Engineering Tool in Real-World AI Systems").
So I'm guessing, even if you get the mechanics of memoization working, it won't really speed it up in this case.
A while ago I wrote a little memoization routine for Scheme that used a chain of closures to keep track of the memoized state:
(define (memoize op)
(letrec ((get (lambda (key) (list #f)))
(set (lambda (key item)
(let ((old-get get))
(set! get (lambda (new-key)
(if (equal? key new-key) (cons #t item)
(old-get new-key))))))))
(lambda args
(let ((ans (get args)))
(if (car ans) (cdr ans)
(let ((new-ans (apply op args)))
(set args new-ans)
new-ans))))))
This needs to be used like so:
(define fib (memoize (lambda (x)
(if (< x 2) x
(+ (fib (- x 1)) (fib (- x 2)))))))
I'm sure that this can be ported to your favorite lexically scoped Lisp flavor with ease.
I'd probably do something like:
(let ((memo (make-hash-table :test #'equal)))
(defun collatz-steps (n)
(or (gethash n memo)
(setf (gethash n memo)
(cond ((= n 1) 0)
((oddp n) (1+ (collatz-steps (+ 1 n n n))))
(t (1+ (collatz-steps (/ n 2)))))))))
It's not Nice and Functional, but, then, it's not much hassle and it does work. Downside is that you don't get a handy unmemoized version to test with and clearing the cache is bordering on "very difficult".