I am writing a program in Common Lisp which needs to store a bunch of status bits for a very large number of entries in an array (the whole program is pretty much fortran-in-lisp) and the status bits are encoded as bits in a fixnum sitting in this array. The accessors for these status bits are actually going to be defined by a macro so I don't have to care about allocating the bits, but a sample reader function might be
(defun deadp (e)
(logbitp 0 e))
(In real life this will be inlined and cluttered with declarations to try and make sure it's quick but those don't matter here I think.)
I need these things to be functions because I want to be able to map them, but also because the using-a-macro-to-inline-a-function thing makes me feel bad.
Then I'll be using this as something like this:
(defconstant status-index 3)
...
(dotimes (i nentries)
(unless (deadp (aref entries i status-index))
...))
(In real life (aref entries i status-index) will be (status entries i), which in turn is going to need a setf method but I think that's easy.)
or
(loop for i below nentries
counting (if (deadp entries i status-index) 1 0))
And of course there will be other similar single-bit flags which will have different bits associated with them.
So, now I want to be able to do this:
(dotimes (i nentries)
...
(when ...
(setf (deadp (aref entries i status-index) t)))
...)
which should turn into code equivalent to
(dotimes (i nentries)
...
(when ...
(progn
(setf (ldb (byte 1 0) (aref entries i status-index)) 1)
t))
...)
And also this:
(let ((status 0))
...
(when ...
(setf (deadp status) t))
...)
which should turn into code equivalent to this:
(let ((status 0))
...
(when ...
(progn
(setf (ldb (byte 1 0) status) 1)
t))
...)
In other words I want my deadp function to be an accessor and for setf on it to work in a general way for it: (setf (deadp (cdr x)) nil) should work, etc.
So this has dropped me into bits of CL that I've avoided for a long time: defining setf expanders. Pretty obviously just defining a (setf deadp) function won't work because numbers are immutable, and I'm fairly sure that defsetf is not powerful enough, so I need define-setf-expander, which I don't understand.
Can someone explain how I need to do this? I think the particular deadp function is not critical, although all the functions I care about will look like variants of it.
An alternative answer would be 'that's a braindead approach, instead do ...', and I'm open to those. I have considered writing code which abstracts the array away, so instead of (deadp (aref ...)) I'd write (deadp people ...) where people is the array of people. This would be fine, and it's easy to see how to make that setfable, except I also want to be able to say (deadp status) where status is just a fixnum. But perhaps there is some better approach.
According to the SBCL documentation for GET-SETF-EXPANSION, a setf expander must:
"Return five values needed by the SETF machinery: a list of temporary
variables, a list of values with which to fill them, a list of temporaries
for the new values, the setting function, and the accessing function."
The setting function and accessing function are actually just the forms that set and access the value at the place, not function objects.
Try this:
(define-setf-expander deadp (place)
(let ((new (gensym)))
(values nil nil (list new)
`(progn (setf (ldb (byte 1 0) ,place) (if ,new 1 0))
,new)
`(deadp ,place))))
Sample expansion:
(let ((status 1))
(setf (deadp status) t))
->
(let ((status 1))
(LET* ((#:G605 T))
(SETF (LDB (BYTE 1 0) STATUS)
(IF #:G605
1
0))
#:G605))
Related
I have a function like this:
(defun lookup-data (index-key)
(let* ((key-table '("key0" "key1" "key2" ...))
(index (position index-key key-table :test #'string-equal))
...
; do stuff with index, among other things
)
The key table (really just a list of strings, but it's being used as a lookup table to map a string to an index number) is a literal value known at read time. I was thinking perhaps it should be made a defparameter or defconstant, but it's not used anywhere outside this one function. I assume the fact that it's a literal means that most compilers can do constant-based optimization on it as-is, but is there something else I should do to mark it as a constant? What are the options here?
Your code is fine.
key-table is a constant, it will be created once, when the function is compiled.
PS. You can also use #. to create more complicated constants that require code:
(defun ... (...)
(let ((unit #.(let ((u (make-array '(10000 10000) :element-type 'double-float
:initial-element 0)))
(dolist (i 10000 unit)
(setf (aref u i i) 1))))
...)
...))
here the unit matrix unit is created at read time and is a constant (well, you can modify it, but...).
Been at this for hours, I'm new to LISP and could not figure out what is wrong with this. I'm trying to solve a matrix problem using backtracking. Can someone help?
(defvar m 3)
(defvar n 4)
(defvar M-DASH (make-array (list m n)
:initial-contents '((1 0 1 0) (0 1 1 0) (1 0 1 1))))
(defvar selected (make-array n))
(defvar set-count 0)
(defun cand-set-count (M-DASH set-count selected curr)
(if (>= curr m)
(progn
(incf set-count)
(return-from cand-set-count 0)
))
(loop for i from 0 to (- n 1)
do (if (and (equal (aref M-DASH curr i) 1) (null (aref selected i)))
(progn
(setf (aref selected i) 1)
(cand-set-count M-DASH set-count selected (+ curr 1))
(setf (aref selected i) nil)
))))
(cand-set-count M-DASH set-count selected 0)
(format t "~A" set-count)
Before answering the actual question, a few notes on style and general remarks:
Special variables (i.e. introduced by defvar or defparameter) are usually named with a name starting and ending with * (for example, (defvar *m* 4)
Unless you did some very weird modifications, the reader won't be case sensitive. This means that M-DASH and m-dash are the same symbol. For this reason, we tend not to use any form of capitalization, as it might induce some mistakes (e.g. believing that two symbols are different while they are in fact identical ...).
If you use a if but only care about the then or the else part, use when/unless instead. It makes intent clearer, and also comes with what is known as an "implicit progn":
(if some-test
(progn
(do-smtg)
(do-smtg-else)))
is identical to
(when some-test
(do-smtg)
(do-smtg-else))
If you are only looping from 0 to some number, and not doing very complex things (or, if you are a beginner, and don't want to deal with the specific syntax rules of the loop macro ...), use dotimes instead. It is more readable, less confusing, and in general less prone to mistake.
With that in mind, your main function can be rewritten as:
(defun cand-set-count (m-dash set-count selected curr)
(when (>= curr m)
(incf set-count)
(return-from cand-set-count 0))
(dotimes (i n)
(when (and (equal (aref m-dash curr i) 1)
(null (aref selected i)))
(setf (aref selected i) 1)
(cand-set-count m-dash set-count selected (+ curr 1))
(setf (aref selected i) nil))))
which is definitely clearer.
And now, the actual problem: the line (incf set-count) does not do what you think it does. It does not increase the value of the special variable named set-count ... and that would be clear, had you followed the convention of naming special variables with * around their name (I am not being sarcastic here: it is (one of ...) the reason conventions exist). What gets incremented is the local binding, not the value of the special variable. You can try it with a simpler code:
* (defvar *foo* 0)
0
* (defun bar (foo)
(incf foo))
BAR
* (bar *foo*)
1
* *foo*
0
Said differently: you have a special (and so, in some sense, global) variable named set-count, but your function also takes an argument, named set-count ! Do you want to go pure functional & side-effect-free, or do you want to mutate state and pass global references around ? Either is fine, but you need to stick to it.
The same is true for the other arguments, m-dash and selected, that are both "shadowed" inside the function body, and never actually modified.
Another mistake that might bite you later on: arrays are not initialized by default. You cannot suppose that (defvar selected (make-array n)) initializes anything to 0 or to nil; your function later tests if (null (aref selected i)), but if you do not explicitly initialize selected with :initial-contents/:initial-element, the result is undefined.
I am working on problem to get the occurence of Prime in a list in lisp.
Input:
Write a function (OccurencesOfPrimes < list >) which counts the number of primes in a (possibly nested) list.
Output: Example: (OccurencesOfPrimes (((1)(2))(5)(3)((8)3)) returns 4.
I am using the below code but getting the error like:
(
defun OccurencesOfPrimes (list)
(loop for i from 2 to 100
do ( setq isPrime t)
(loop for j from 2 to i
never (zerop (mod i j))
(setq isPrime f)
(break)
)
)
(if (setq isPrime t)
(append list i)
)
)
)
LOOP: illegal syntax near (SETQ ISPRIME F) in
(LOOP FOR J FROM 2 TO I NEVER (ZEROP (MOD I J)) (SETQ ISPRIME F) (BREAK)
)
Any help.
It is important to keep the format consistent with the expected conventions of the language. It helps when reading the code (in particular with other programmers), and can help you see errors.
Also, you should use an editor which, at the minimum, keep tracks of parentheses. In Emacs, when you put the cursor in the first opening parenthesis, the matching parenthesis is highlighted. You can spot that you have one additional parenthesis that serves no purpose.
(
defun OccurencesOfPrimes (list)
(loop for i from 2 to 100
do ( setq isPrime t)
(loop for j from 2 to i
never (zerop (mod i j))
(setq isPrime f)
(break)
)
)
(if (setq isPrime t)
(append list i)
)
) ;; <- end of defun
) ;; <- closes nothing
In Lisp, parentheses are for the computer, whereas indentation is for humans. Tools can automatically indent the code according to the structure (the parenthesis), and any discrepancy between what indentation you expect and the one being computed is a hint that your code is badly formed. If you look at the indentation of your expressions, you can see how deep you are in the form, and that alone helps you understand the code.
Symbol names are dash-separated, not camlCased.
Your code, with remarks:
(defun occurences-of-primes (list)
;; You argument is likely to be a LIST, given its name and the way
;; you call APPEND below. But you never iterate over the list. This
;; is suspicious.
(loop
for i from 2 to 100
do
(setq is-prime t) ;; setting an undeclared variable
(loop
for j from 2 to i
never (zerop (mod i j))
;; the following two forms are not expected here according
;; to LOOP's grammar; setting IS-PRIME to F, but F is not
;; an existing variable. If you want to set to false, use
;; NIL instead.
(setq is-prime f)
;; BREAK enters the debugger, maybe you wanted to use
;; LOOP-FINISH instead, but the NEVER clause above should
;; already be enough to exit the loop as soon as its
;; sub-expression evaluates to NIL.
(break)))
;; The return value of (SETQ X V) is V, so here your test would
;; always succeed.
(if (setq is-prime t)
;; Append RETURNS a new list, without modifying its
;; arguments. In particular, LIST is not modified. Note that "I"
;; is unknown at this point, because the bindings effective
;; inside the LOOP are not visible in this scope. Besides, "I"
;; is a number, not a list.
(append list i)))
Original question
Write one function which counts all the occurrences of a prime number in a (possibly nested) list.
Even though the homework questions says "write one function", it does not say that you should write one big function that compute everything at once. You could write one such big function, but if you split your problem into sub-problems, you will end with different auxiliary functions, which:
are simpler to understand (they do one thing)
can be reused to build other functions
The sub-problems are, for example: how to determine if a number is a prime? how to iterate over a tree (a.k.a. a possibly nested list)? how to count
the occurrences?
The basic idea is to write an "is-prime" function, iterate over the tree and call "is-prime" on each element; if the element is prime and was never seen before, add 1 to a counter, local to your function.
You can also flatten the input tree, to obtain a list, then sort the resulting
list; you iterate over the list while keeping track of the last
value seen: if the value is the same as the previous one, you
already know if the number is prime; if the previous number differs, then
you have to test if the number is prime first.
You could also abstract things a little more, and define a higher-order tree-walker function, which calls a function on each leaf of the tree. And write another higher-order function which "memoizes" calls: it wraps around a
function F so that if you call F with the same arguments as before,
it returns the result that was stored instead of recomputing it.
Example
I'll combine the above ideas because if you give that answer to a teacher you are likely to have to carefully explain what each part does (and if you can, great for you); this is not necessarily the "best" answer, but it covers a lot of things.
(defun tree-walk-leaves (tree function)
(typecase tree
(null nil)
(cons
(tree-walk-leaves (car tree) function)
(tree-walk-leaves (cdr tree) function))
(t (funcall function tree))))
(defun flatten (tree &optional keep-order-p)
(let ((flat nil))
(tree-walk-leaves tree (lambda (leaf) (push leaf flat)))
(if keep-order-p
(nreverse flat)
flat)))
(defun prime-p (n)
(or (= n 2)
(and (> n 2)
(oddp n)
(loop
for d from 3 upto (isqrt n) by 2
never (zerop (mod n d))))))
(defun count-occurences-of-prime (tree)
(count-if #'prime-p (remove-duplicates (flatten tree))))
(count-occurences-of-prime '(((1)(2))(5)(3)((8)3)))
=> 4
If, instead, you don't want to remove duplicates but count the multiple times a prime number occurs, you can do:
(count-if (memoize #'prime-p) (flatten tree))
... where memoize is:
(defun memoize (function &key (test #'equalp) (key #'identity))
(let ((hash (make-hash-table :test test)))
(lambda (&rest args)
(let ((args (funcall key args)))
(multiple-value-bind (result exists-p) (gethash args hash)
(values-list
(if exists-p
result
(setf (gethash args hash)
(multiple-value-list (apply function args))))))))))
(memoize is useless if there are no duplicates)
In Python you can write this:
def firstn(n):
num = 0
while num < n:
yield num
num += 1
What is the lisp equivalent of this?
Existing package
Download, install and load the GENERATORS system with Quicklisp. Then, use package :generators (or preferably, define your own package first).
(ql:quickload :generators)
(use-package :generators)
Define an infinite generator for random values:
(defun dice (n)
(make-generator ()
;; repeatedly return a random value between 1 and N
(loop (yield (1+ (random n))))))
Use the generator:
(loop
with dice = (dice 6)
repeat 20
collect (next dice))
=> (1 2 6 1 1 4 4 2 4 3 6 2 1 5 6 5 1 5 1 2)
Note however what the author of the library says:
This library is more of an interesting toy, though as far as I know it
does work. I dont think I have ever used this in application code,
though I think that with care, it could be.
See also
The ITERATE package provides a way to define generators for use inside its iteration facility.
The SERIES package provide stream-like data structures and operations on them.
The Snakes library (same approach as GENERATORS as far as I know).
Iterators in generic-cl
Closures
In practice, CL does not rely that much on generators as popularized by Python. What happens instead is that when people need lazy sequences, they use closures:
(defun dice (n)
(lambda ()
(1+ (random n))))
Then, the equivalent of next is simply a call to the thunk generated by dice:
(loop
with dice = (dice 6)
repeat 20
collect (funcall dice))
This is the approach that is preferred, in particular because there is no need to rely on delimited continuations like with generators. Your example involves a state, which the dice example does not require (there is a hidden state that influences random, but that's another story) . Here is how your counter is typically implemented:
(defun first-n (n)
(let ((counter -1))
(lambda ()
(when (< counter n)
(incf counter)))))
Higher-order functions
Alternatively, you design a generator that accepts a callback function which is called by your generator for each value. Any funcallable can be used, which allows the caller to retain control over code execution:
(defun repeatedly-throw-dice (n callback)
(loop (funcall callback (1+ (random n)))))
Then, you can use it as follows:
(prog ((counter 0) stack)
(repeatedly-throw-dice 6
(lambda (value)
(if (<= (incf counter) 20)
(push value stack)
(return (nreverse stack))))))
See documentation for PROG.
do-traversal idiom
Instead of building a function, data sources that provides a custom way of generating values (like matches of a regular expressions in a string) also regularly provide a macro that abstracts their control-flow. You would use it as follows:
(let ((counter 0) stack)
(do-repeatedly-throw-dice (value 6)
(if (<= (incf counter) 20)
(push value stack)
(return (nreverse stack))))))
DO-X macros are expected to define a NIL block around their body, which is why the return above is valid.
A possible implementation for the macro is to wrap the body in a lambda form and use the callback-based version defined above:
(defmacro do-repeatedly-throw-dice ((var n) &body body)
`(block nil (repeatedly-throw-dice ,n (lambda (,var) ,#body))))
Directly expanding into a loop would be possible too:
(defmacro do-repeatedly-throw-dice ((var n) &body body)
(let ((max (gensym)) (label (make-symbol "NEXT")))
`(prog ((,max ,n) ,var)
,label
(setf ,var (1+ (random ,max)))
(progn ,#body)
(go ,label))))
One step of macroexpansion for above form:
(prog ((#:g1078 6) value)
#:next
(setf value (1+ (random #:g1078)))
(progn
(if (<= (incf counter) 20)
(push value stack)
(return (nreverse stack))))
(go #:next))
Bindings
Broadly speaking, building a generator with higher-order functions or directly with a do- macro gives the same result. You can implement one with the other (personally, I prefer to define first the macro and then the function using the macro, but doing the opposite is also interesting, since you can redefine the function without recompiling all usages of the macro).
However, there is still a difference: the macro reuses the same variable across iterations, whereas the closure introduces a fresh binding each time. For example:
(let ((list))
(dotimes (i 10) (push (lambda () i) list))
(mapcar #'funcall list))
.... returns:
(10 10 10 10 10 10 10 10 10 10)
Most (if not all) iterators in Common Lisp tend to work like this1, and it should not come as a surprise for experienced users (the opposite would be surprising, in fact). If dotimes was implemented by repeatedly calling a closure, the result would be different:
(defmacro my-dotimes ((var count-form &optional result-form) &body body)
`(block nil
(alexandria:map-iota (lambda (,var) ,#body) ,count-form)
,result-form))
With the above definition, we can see that:
(let ((list))
(my-dotimes (i 10) (push (lambda () i) list))
(mapcar #'funcall list))
... returns:
(9 8 7 6 5 4 3 2 1 0)
In order to have the same result with the standard dotimes, you only need to create a fresh binding before building the closure:
(let ((list))
(dotimes (i 10)
(let ((j i))
(push (lambda () j) list))))
Here j is a fresh binding whose value is the current value of i at closure creation time; j is never mutated so the closure will constantly return the same value.
If you wanted to, you could always introduce that inner let from the macro, but this is rarely done.
1: Note that the specification for DOTIMES does not require that bindings are fresh at each iteration, or only mutates the same binding at each step: "It is implementation-dependent whether dotimes establishes a new binding of var on each iteration or whether it establishes a binding for var once at the beginning and then assigns it on any subsequent iterations." In order to write portably, it is necessary to assume the worst-case scenario (i.e. mutation, which happens to be what most (all?) implementations do) and manually rebind iteration variables if they are to be captured and reused at a later point.
We've been using Lisp in my AI course. The assignments I've received have involved searching and generating tree-like structures. For each assignment, I've ended up writing something like:
(defun initial-state ()
(list
0 ; score
nil ; children
0 ; value
0)) ; something else
and building my functions around these "states", which are really just nested lists with some loosely defined structure.
To make the structure more rigid, I've tried to write accessors, such as:
(defun state-score ( state )
(nth 2 state))
This works for reading the value (which should be all I need to do in a nicely functional world. However, as time crunches, and I start to madly hack, sometimes I want a mutable structure). I don't seem to be able to SETF the returned ...thing (place? value? pointer?).
I get an error with something like:
(setf (state-score *state*) 10)
Sometimes I seem to have a little more luck writing the accessor/mutator as a macro:
(defmacro state-score ( state )
`(nth 2 ,state))
However I don't know why this should be a macro, so I certainly shouldn't write it as a macro (except that sometimes it works. Programming by coincidence is bad).
What is an appropriate strategy to build up such structures?
More importantly, where can I learn about whats going on here (what operations affect the memory in what way)?
Use CLOS for data structures
The best way out of this is to quickly learn the basics of CLOS.
(defclass state ()
((score :accessor state-score :initform 0)
(children :accessor state-children :initform nil)
(value :accessor state-value :initform 0)))
(defun make-initial-state ()
(make-instance 'state))
(defparameter *state* (make-initial-state))
(setf (state-score *state*) 10)
For most application code avoid structures
For most code avoid structures - use them only when you need them and know why. Use CLOS classes instead.
DEFSTRUCT also works with lists
If you really want to use lists, one option is to use the DEFSTRUCT macro with lists and have it define all the functions:
(defstruct (state (:type list))
(score 0)
(children nil)
(value 0))
Above, the :type option tells DEFSTRUCT to use a list instead a structure.
? (defparameter *state* (make-state))
*STATE*
? (setf (state-score *state*) 10)
10
(make-state) returns a list of three items.
We can write setter functions
If you want to write code by hand, then you can write setter functions:
(defun get-my-score (state)
(first state))
(defun (setf get-my-score) (score state)
(setf (first state) score))
Above defines a SETF function. The name of the function is actually a list. The parameters for this function have to be first the new-value and then the thing to set.
? (setf *state* (list 0 nil 0))
(0 NIL 0)
? (setf (get-my-score *state*) 10)
10
? *state*
(10 NIL 0)
The Common Lisp HyperSpec defines what places are and how to work with them. I would guess that this is not the best source to learn from and possibly it is best explained in some introductory Lisp book.
You can use something like this:
(defun get-score (state)
(nth 0 state)) ; This corresponds to the comments in the init function
(defun set-score (state new-value)
(setf (nth 0 state) new-value))
(defsetf get-score set-score)
This way, any time you write (setf (get-score something) else) it will be translated to (set-score something else).
Use defstruct:
> (defstruct state score children val something-else)
STATE
> (setq initial-state (make-state :score 0 :children nil :val 0 :something-else nil))
#S(STATE :SCORE 0 :CHILDREN NIL :VAL 0 :SOMETHING-ELSE NIL)
> (state-score initial-state) ; current score
0
> (setf (state-score initial-state) 10) ; set new score
10
> (state-score initial-state)
10
This happens because setf is a macro. When you define state-score as a macro, setf sees:
(setf (nth 2 state) value)
And knows what to do since it can use nth as a place to store values. On the other hand, when state-score is a function, setf just sees a value returned, and can't do anything about it.
Read more about how setf works and its concept of places for a deeper understanding. Here's an interesting tutorial that says:
The setf special form uses its first
argument to define a place in memory,
evaluates its second argument, and
stores the resulting value in the
resulting memory location