Found this while looking for a memoizing function:
#lang scheme
(define (memo f)
(define mh (make-hash))
(lambda p
(hash-ref mh p (lambda ()
(hash-set! mh p (apply f p))
(hash-ref mh p)))))
(define-syntax-rule (defmemo (id . p) . body)
(define id (memo (lambda p . body))))
(provide defmemo)
as seen here: Writing an auto-memoizer in Scheme. Help with macro and a wrapper
However in the documentation there's also: make-weak-hash and make-immutable-hash. Which hash function is better for memoization?
Great question.
I would say for memoization sticking with make-hash is your best bet. The other options are used for different purposes.
make-immutable-hash is for making, well, you guessed it, immutable hash tables. The hash tables it creates are the same as if you had created them with hash. Of course, the items 'in' the table can still be mutable, such as if you put a mutable struct in there. Its just that the table itself is not mutable.
make-hash is used for making mutable hash tables. This is probably what you want here since you are mutating your memoization table.
make-weak-hash is actually completely different. Specifically, it stores 'weak' pointers to the items in the table. That is, it does not count as a pointer as far as the garbage collector is concerned. So, if you use them, your items may end up being collected. (This doesn't mean you'll get a segfault, you just may lose your data.) Thus, you generally don't want this table unless you know you specifically want this behavior.
Related
I've been playing with LISP lately, and I want to attempted to optimize a terribly inefficient recursive function by caching it's output so it only runs each combination of parameters once. I want to save the results into hash table, using either (LIST a b) or (CONS a b) as the key. My question is, is that even possible? Documentation I've read would lead me to believe it is, as the document I read defines the key as an 'object', and defines 'object' as something that was constructed with cons. So I gave this a try.
(defparameter *table* (make-hash-table))
(setf (gethash (list 1 2) *table*) 123)
(format t "~S~%" (gethash (list 1 2) *table*))
And it gives me NIL, when I would expect 123. If i replace the (list 1 2) with a cons it still doesn't work, but when I use an plain integer, it works fine.
I'm using GCL 2.6.12
(make-hash-table :test #'equal)
Solutions for caching results of functions has been already implemented for Common Lisp. One is library fare-memoization, the second is function-cache and both are accessible with the Quicklisp. Using the former is as easy as defining normal function:
(define-memo-function your-time-consuming-function (arg)
(code arg))
For reference please check https://github.com/fare/fare-memoization.
It seems like in order to use multiple return values in Racket, I have to either use define-values or collect them into a list with (call-with-values (thunk (values-expr)) list). In the latter case, why would someone to choose to return multiple values instead of a list, if just have to collect them into a list anyway? Additionally, both of these are very wordy and awkward to work into most code. I feel like I must be misunderstanding something very basic about multiple-return-values. For that matter, how do I write a procedure accepting multiple return values?
Although I may be missing some of the Scheme history and other nuances, I'll give you my practical answer.
First, one rule of thumb is if you need to return more than 2 or 3 values, don't use multiple values and don't use a list. Use a struct. That will usually be easier to read and maintain.
Racket's match forms make it much easier to destructure a list return value -- as easy as define-values:
(define (f)
(list 1 2))
(match-define (list a b) (f))
(do-something-with a b)
;; or
(match (f)
[(list a b) (do-something-with a b)])
If you have some other function, g, that takes a (list/c a b), and you want to compose it with f, it's simpler if f returns a list. It's also simpler if both use a two-element struct. Whereas call-with-values is kind of an awkward hot mess, I think.
Allowing multiple return value is an elegant idea, because it makes return values symmetric with arguments. Using multiple values is also faster than lists or structs (in the current implementation of Racket, although it could work otherwise).
However when readability is a higher priority than performance, then in modern Racket it can be more practical to use a list or a struct, IMHO. Having said that I do use multiple values for one-off private helper functions.
Finally, there's a long, interesting discussion on the Racket mailing list.
Racket doc gives us the quintessential example why, in disguise:
> (let-values ([(q r) (quotient/remainder 10 3)])
(if (zero? r)
q
"3 does *not* divide 10 evenly"))
"3 does *not* divide 10 evenly"
We get two values directly, and use them separately in a computation that follows.
update: In Common Lisp, with its decidedly practical, down-to-the-metal, non-functional approach (where they concern themselves with each extra cons cell allocation), it makes much more sense, especially as it allows one to call such procedures in a "normal" way as well, automatically ignoring the "extra" results, kind of like
(let ([q (quotient/remainder 10 3)])
(list q))
But in Racket this is invalid code. So yeah, it looks like an extraneous feature, better to be avoided altogether.
Using list as the consumer defeats the purpose of multiple values so in that case you could just have used lists to begin with. Multiple values is actually a way of optimization.
Semanticly returning a list and several values are similar, but where you return many values in a list effort goes into creation of cons cells to make the list and destructuring accessors to get the values at the other end. In many cases however, you wouldn't notice the difference in performance.
With multiple values the values are on the stack and (call-with-values (lambda () ... (values x y z)) (lambda (x y z) ...) only checks the number to see if it's correct.. If it's ok you just apply the next procedure since the stack has it's arguments all set from the previous call.
You can make syntactic sugar around this and some popular ones are let-values and SRFI-8 receive is a slightly simpler one. Both uses call-with-values as primitive.
values is handy because it
checks that the number of elements returned is correct
destructures
For example, using
(define (out a b) (printf "a=~a b=~a\n" a b))
then
(let ((lst (list 1 2 3)))
(let ((a (first lst)) (b (second lst))) ; destructure
(out a b)))
will work even though lst has 3 elements, but
(let-values (((a b) (values 1 2 3)))
(out a b))
will not.
If you want the same control and destructuring with a list, you can however use match:
(let ((lst (list 1 2)))
(match lst ((list a b) (out a b))))
Note that he creation of the structure, e.g. (list 1 2) vs (values 1 2) is equivalent.
Probably the hardest part of learning lisp has been to think in the "lisp way" which is elegant and impressive, but not always easy. I know that recursion is used to solve a lot of problems, and I am working through a book that instead uses apply to solve a lot of problems, which I understand is not as lispy, and also not as portable.
An experienced lisper should be able to help with this logic without knowing specifically what describe-path location and edges refer to. Here is an example in a book I am working through:
(defun describe-paths (location edges)
(apply (function append) (mapcar #'describe-path
(cdr (assoc location edges)))))
I have successfully rewritten this to avoid apply and use recursion instead. It seems to be working:
(defun describe-paths-recursive (location edges)
(labels ((processx-edge (edge)
(if (null edge)
nil
(append (describe-path (first edge))
(processx-edge (rest edge))))))
(processx-edge (cdr (assoc location edges)))))
I would like some more seasoned pairs of eyes on this to advise if there is a more elegant way to translate the apply to recursion, or if I have done something unwise. This code seems decent, but would there been something even more "lispy" ?
(apply (function append) (mapcar #'g ...)) is just mapcan (update: with usual caveats about destructive update and quoted lists, see also this):
(defun describe-paths (location edges)
(mapcan #'describe-path
(cdr (assoc location edges))))
Recursion is good for thinking, for understanding. But actually using it in your code comes with a price.
Your recursive re-write is tail recursive modulo cons; no Lisp has this optimization AFAIK, even though it was first described in 1974, in Lisp.
So what you wrote is good as an executable specification.
But Common Lisp is a practical language. In particular, it has many ways to encode iteration. Remember, iterative processes are our goal; recursive processes are terrible, efficiency-wise. So when we write a code which is syntactically recursive, we still want it to describe an iterative process (such that runs in constant stack space).
Common Lisp, being a practical language, would have us just write the loop out directly. For one,
(defun describe-paths-loop (location edges &aux (res (list 1)) (p res))
(dolist (x (cdr (assoc location edges))
(cdr res)) ; the return form
(setf (cdr p) (describe-path x))
(setf p (last p))))
is guaranteed to work in constant stack space.
update: this destructively concatenates lists returned by describe-path so it should take care not to return lists with the same last cons cell on separate invocations, or this could create circular structure. Alternatively, the call to describe-path could be wrapped in a copy-list call. Of course, if describe-path were to return a list which is already cyclic, last here would go into a loop too.
I saw several opinions about using apply is a bad style. But actually that would be great if somebody will explain me why apply is considered to be bad.
What do you mean using a word "lispy". Common lisp allows to program in any style you want.
If "lispy" means functional programming style, then the first code is written in more functional programming style. A function is passed to a function mapcar and another function is passed to apply and all the job is done by passing the results of one function to another. In you code you don't pass functions as arguments to other functions. But recursion can be considered as functional programming style sign. And code in the book is shorter than yours.
If you don't like apply because of apply determines the argument count in runtime, you can use reduce in this situation (if I understood the data structures correctly):
(Thanks to Joshua Taylor for pointing a huge resource overhead without :from-end t key argument)
(defun describe-paths (location edges)
(reduce #'append (mapcar #'describe-path
(rest (assoc location edges))) :from-end t))
Anyway I'm pretty sure that the purpose of the code in the book is the education reason. It's an example of mapcar and apply that shows how lists are treated as data and code in lisp.
p.s. Actually I figured why apply can be bad (stack is used for function calls).
> (apply #'+ (make-list 500000 :initial-element 1))
*** - Lisp stack overflow. RESET
So as Rainer Joswig told it's lispy to avoid stack overflows. Reduce fix the problem.
> (reduce #'+ (make-list 50000000 :initial-element 1))
50000000
The Lisp way is to use functional, imperative or object-oriented programming (with or without mutable state) to solve a problem, or to invent some other programming as you see fit and express it in macros. Looking for recursion while ignoring other approaches is not the Lisp way; it's the way of the wayward Lisp academic.
The most straightforward way to rewrite the function:
(defun describe-paths (location edges)
(apply (function append) (mapcar #'describe-path
(cdr (assoc location edges)))))
is to use loop. The proper motivation for eliminting apply is that we expect many paths, which could exceed the limit on the number of arguments to a function.
All you are doing with apply is making a big argument list to the append function. We can append any number of lists into a big list with loop like this:
(defun describe-paths (location edges)
(loop for path in (cdr (assoc location edges))
appending (describe-path path))
Presumably, describe-path returns a list, and you want to catenate these together.
The appending clause of loop, which may also be spelled append, gathers appends the value of is argument form into an anonymous list. That list becomes the return value when the loop terminates.
We can use nconcing to improve the performance if we have justification in believing that the lists returned by described-path are freshly allocated on each call.
There's nothing wrong with this question; plenty of questions similar to this are asked in the python category, for example.
But to your question: what you are doing is Good. In fact, it closely resembles, nearly identically, a more general technique Peter Norvig shows in one of his Lisp books, so either you've read that book, or you stumbled upon a good practice on your own. Either way, this is a perfectly acceptable implementation of recursion.
I wonder if there is a less verbose way than using a loop. Anyway this works for me in CLISP:
(loop for key being the hash-keys of *my-hash* collect key)
I've seen others using maphash, but that involves accumulating each key into a list. Apart from that being more involved than using loop, it also introduces a side-effect, which I try to avoid whenever I can - I prefer functional programming as much as possible :)
Is there anything predefined like this for this common task, even though implementation-specific?
(defun hash-keys (hash-table)
(loop for key being the hash-keys of hash-table collect key))
Common Lisp comes from before "batteries-included" philosophy became prevalent, and most functionality is expected to be provided by third-party libraries and not implementations. While Common Lisp is sometimes called a large language, it is only compared to C and similar, the language itself is quite tiny compared to Python and other languages with massive standard libraries.
For this specific purpose, Alexandria is a commonly used collection of Common Lisp utilities. Among many other things it contains hash-table-keys.
There is no disadvantage in defining
(defun hash-keys (hash-table)
(loop for key being the hash-keys of hash-table collect key))
because in Common Lisp the function is compiled. If your vendor provided this function, it would pretty much do the same thing and not be much more efficient than yours, if at all.
In interpreted languages, nearly anything you write yourself has a performance disadvantage compared to an "intrinsic" routine.
Consing up the contents of a hash is wasteful; loop lets you process the hash without consing up memory. So maybe you want a macro instead (Some Lisps provide a dohash or similar as an extension).
(defmacro do-hash ((key-var val-var hash-expr &optional result-form) &body body)
(let ((hash-var (gensym "HASH-")))
`(loop with ,hash-var = ,hash-expr
for ,key-var being the hash-keys of ,hash-var
for ,val-var being the hash-values of ,hash-var
do (progn ,#body)
finally (return ,result-form))))
Or a hash mapping function:
(defun mapc-hash (hash-table fun)
(loop for key being the hash-keys of hash-table
for value being the hash-values of hash-table
do (funcall fun key value)))
Should the language have every possible gadget like this that anyone can write in a minute?
In Common Lisp, there are included batteries, but they are other kinds of batteries: things that are actually hard to do. For example, a compile function for dynamically compiling code at run time. It would be prohibitively difficult for most users to develop such a thing from scratch compared to pulling keys or values from a hash table in half a dozen different ways.
If you plan to use it more than once, the less lengthy solution is, to write yourself a very short helper function:
(defun hash-table-keys (ht)
(let ((keys nil))
(maphash
#'(lambda (k v)
(declare (ignore v))
(push k keys))
ht)
keys))
As you can see and as other answers have also pointed out, in many cases, it is easier to iterate all entries of a hash table, rather than constructing a filtered list and then iterating over that one.
Using the standard Common Lisp function maphash is useful in both scenarios.
Consider this piece of code:
(defvar lst '(1 1))
(defmacro get-x (x lst)
`(nth ,x ,lst))
(defun get-y (y lst)
(nth y lst))
Now let us assume that I want to change the value of the elements of the list called lst, the car with get-x and the cdr with get-y.
As I try to change the value with get-x (with setf) everything goes fine but if I try it with get-y it signals an error (shortened):
; caught STYLE-WARNING:
; undefined function: (SETF GET-STUFF)
Why does this happen?
I myself suspect that this happens because the macro simply expands and the function nth simply returns a reference to the value of an element in the list and the function on the other hand evaluates the function-call to nth and returns the value of the referenced value (sounds confusing).
Am I correct in my suspicions?
If I am correct then how will one know what is simply a reference to a value and an actual value?
The error does not happen with the macro version, because, as you assumed, the expression (setf (get-x some-x some-list) some-value) will be expanded (at compile-time) into something like (setf (nth some-x some-list) some-value) (not really, but the details of setf-expansion are complex), and the compiler knows, how to deal with that (i.e., there is a suitable setf expander defined for function nth).
However, in the case of get-y, the compiler has no setf expander, unless you provide one. The easiest way to do so would be
(defun (setf get-y) (new-value x ls) ; Note the function's name: setf get-y
(setf (nth x ls) new-value))
Note, that there are a few conventions regarding setf-expanders:
The new value is always provided as the first argument to the setf function
All setf functions are supposed to return the new value as their result (as this is, what the entire setf form is supposed to return)
There is, BTW, no such concept as a "reference" in Common Lisp (at least not in the C++ sense), though there once were Lisp dialects which had locatives. Generalized place forms (ie., setf and its machinery) work very differently from plain C++ style references. See the CLHS, if you are curious about the details.
SETF is a macro.
The idea is that to set and read elements from data structures are two operations, but usually require two different names (or maybe even something more complex). SETF now enables you to use just one name for both:
(get-something x)
Above reads a datastructure. The inverse then simply is:
(setf (get-something x) :foobar)
Above sets the datastructure at X with :FOOBAR.
SETF does not treat (get-something x) as a reference or something like that. It just has a database of inverse operations for each operation. If you use GET-SOMETHING, it knows what the inverse operation is.
How does SETF know it? Simple: you have to tell it.
For The NTH operation, SETF knows how to set the nth element. That's builtin into Common Lisp.
For your own GET-Y operation SETF does not have that information. You have to tell it. See the Common Lisp HyperSpec for examples. One example is to use DEFUN and (SETF GET-Y) as a function name.
Also note following style problems with your example:
lst is not a good name for a DEFVAR variable. Use *list* as a name to make clear that it is a special variable declared by DEFVAR (or similar).
'(1 2) is a literal constant. If you write a Common Lisp program, the effects of changing it are undefined. If you want to change a list later, you should cons it with LIST or something like COPY-LIST.