speeding up deleting duplicates when they're adjacent - lisp

I'm looking for something like #'delete-duplicates, but I know that all elements of the list are already sorted, or inversely sorted, or at least arranged so that duplicates will already be adjacent to each other. I wish to use that knowledge to ensure that execution speed is not proporational to the square of the number of elements in the list. It's trivial to use #'maplist to grow my own solution, but is there something already in the language? It would be embarrassing to reinvent the wheel.
To be clear, for largish lengths of lists, I would like the running time of the deletion to be proportional to the length of the list, not proportional to the square of that length. This is the behavior I wish to avoid:
1 (defun one-shot (cardinality)
2 (labels ((generate-list (the-count)
3 (let* ((the-list (make-list the-count)))
4 (do ((iterator 0 (1+ iterator)))
5 ((>= iterator the-count))
6 (setf (nth iterator the-list) iterator))
7 the-list)))
8 (let* ((given-list (generate-list cardinality))
9 (stripped-list)
10 (start-time)
11 (end-time))
12 (setf start-time (get-universal-time))
13 (setf stripped-list (delete-duplicates given-list :test #'eql))
14 (setf end-time (get-universal-time))
15 (princ "for n = ")
16 (princ cardinality)
17 (princ ", #'delete-duplicates took ")
18 (princ (- end-time start-time))
19 (princ " seconds")
20 (terpri))))
21 (one-shot 20000)
22 (one-shot 40000)
23 (one-shot 80000)
for n = 20000, #'delete-duplicates took 6 seconds
for n = 40000, #'delete-duplicates took 24 seconds
for n = 80000, #'delete-duplicates took 95 seconds

There's nothing like this in the language, but something like this makes just one pass through the list:
(defun delete-adjacent-duplicates (list &key key (test 'eql))
(loop
for head = list then (cdr head)
until (endp head)
finally (return list)
do (setf (cdr head)
(member (if (null key) (car head)
(funcall key (car head)))
(cdr head)
:key key :test-not test))))
As, #wvxvw pointed out, it might be possible to simplify this iteration using (loop for head on list finally (return list) do ...). However, 3.6 Traversal Rules and Side Effects says that modifying the cdr chain of a list during an object-traversal leads to undefined behavior. However, it's not clear whether loop for head on list is technically an object-traversal operation or not. The documentation about loop says in 6.1.2.1.3 The for-as-on-list subclause that
In the for-as-on-list subclause, the for or as construct iterates over
a list. … The
variable var is bound to the successive tails of the list in form1. At
the end of each iteration, the function step-fun is applied to the
list; the default value for step-fun is cdr. … The for or as construct
causes termination when the end of the list is reached.
This says that the step function is always applied at the end of the iteration, so it sounds like loop for head on list should be OK. At any rate, any possible issues could be avoided by using do loop instead:
(defun delete-adjacent-duplicates (list &key key (test 'eql))
(do ((head list (cdr head)))
((endp head) list)
(setf (cdr head)
(member (if (null key) (car head)
(funcall key (car head)))
(cdr head)
:key key :test-not test))))
The idea is to start with head being the list, then setting its cdr to the first tail that starts with a different element, then advancing the head, and continuing until there's nothing left. This should be linear in the length of the list, assuming that member is implemented in a sensible way. The use of member means that you don't have to do any extra work to get :key and :test working in the appropriate way. (Do note that :test for del-dups is going to be the :test-not of member.) Note: there's actually a slight issue with this, in that the key function will called twice for each element in the final list: once when it's the first element of a tail, and once when it's the car of head.
CL-USER> (delete-adjacent-duplicates (list 1 1 1 1 2 2 3 3 3))
(1 2 3)
CL-USER> (delete-adjacent-duplicates (list 1 2 2))
(1 2)
CL-USER> (delete-adjacent-duplicates (list 1 3 5 6 4 2 3 5) :key 'evenp)
(1 6 3)
I expect that any linear time solution is going to take a similar approach; hold a reference to the current head, find the next tail that begins with a different element, and then make that tail the cdr of the head.

I would expect REMOVE-DUPLICATES to have a linear time implementation. (And indeed it does* on my local SBCL install.)
Note that REMOVE-DUPLICATES and DELETE-DUPLICATES are specified to have the same return value, and that the side effects of DELETE-DUPLICATES are not guaranteed.
* The linear time code path is only taken when the :test is #'eq,#'eql, #'equal, or #'equalp (it relies on a hash table) and there is no :key or :test-not argument supplied.

For the record: your test code is basically just this:
(defun one-shot (n &aux (list (loop for i below n collect i)))
(time (delete-duplicates list))
(values))
It might also be useful to talk to the implementation maintainers in the case of a slow delete-duplicates.
For example (one-shot 1000000) runs in a second in CCL on my Mac. In LispWorks it runs in 0.155 seconds.

There is nothing like that in the language standard. However, you can do that either with a loop:
(defun remove-adjacent-duplicates (list &key (test #'eql))
(loop for obj in list
and prev = nil then obj
for take = t then (not (funcall test obj prev))
when take collect obj))
or with reduce (exercise left to the reader).
See the other answer for a destructive implementation.
PS. Unless you are doing something tricky with timing, you are much better off using time.

A bit different approach:
(defun compress-duplicates (list &key (test #'eql))
(labels ((%compress-duplicates (head tail)
(if (null tail)
(setf (cdr head) tail)
(progn (unless (funcall test (car head) (car tail))
(setf (cdr head) tail head (cdr head)))
(%compress-duplicates head (cdr tail))))))
(%compress-duplicates list (cdr list))
list))
(compress-duplicates (list 1 1 1 2 2 3 4 4 1 1 1))
;; (1 2 3 4 1)
Test of SBCL delete-duplicates implementation:
(defun test-delete-duplicates ()
(labels ((%test (list)
(gc)
(time (delete-duplicates list))))
(loop
:repeat 6
:for list := (loop :for i :from 0 :below 1000
:collect (random 100))
:then (append list list) :do (%test (copy-list list)))))
;; (test-delete-duplicates)
;; Evaluation took:
;; 0.002 seconds of real time
;; 0.002000 seconds of total run time (0.002000 user, 0.000000 system)
;; 100.00% CPU
;; 3,103,936 processor cycles
;; 0 bytes consed
;; Evaluation took:
;; 0.003 seconds of real time
;; 0.003000 seconds of total run time (0.003000 user, 0.000000 system)
;; 100.00% CPU
;; 6,347,431 processor cycles
;; 0 bytes consed
;; Evaluation took:
;; 0.006 seconds of real time
;; 0.006000 seconds of total run time (0.005000 user, 0.001000 system)
;; 100.00% CPU
;; 12,909,947 processor cycles
;; 0 bytes consed
;; Evaluation took:
;; 0.012 seconds of real time
;; 0.012000 seconds of total run time (0.012000 user, 0.000000 system)
;; 100.00% CPU
;; 25,253,024 processor cycles
;; 0 bytes consed
;; Evaluation took:
;; 0.023 seconds of real time
;; 0.022000 seconds of total run time (0.022000 user, 0.000000 system)
;; 95.65% CPU
;; 50,716,442 processor cycles
;; 0 bytes consed
;; Evaluation took:
;; 0.049 seconds of real time
;; 0.050000 seconds of total run time (0.050000 user, 0.000000 system)
;; 102.04% CPU
;; 106,747,876 processor cycles
;; 0 bytes consed
Shows linear speed.
Test of ECL delete-duplicates implementation:
;; (test-delete-duplicates)
;; real time : 0.003 secs
;; run time : 0.003 secs
;; gc count : 1 times
;; consed : 95796160 bytes
;; real time : 0.007 secs
;; run time : 0.006 secs
;; gc count : 1 times
;; consed : 95874304 bytes
;; real time : 0.014 secs
;; run time : 0.014 secs
;; gc count : 1 times
;; consed : 95989920 bytes
;; real time : 0.028 secs
;; run time : 0.027 secs
;; gc count : 1 times
;; consed : 96207136 bytes
;; real time : 0.058 secs
;; run time : 0.058 secs
;; gc count : 1 times
;; consed : 96617536 bytes
;; real time : 0.120 secs
;; run time : 0.120 secs
;; gc count : 1 times
;; consed : 97412352 bytes
Linear time increase too.

Related

Simplifying a Racket function

I have the following function "change" which takes a certain amount of money to be paid, the size of the bill/coin used to pay, and returns a list with the number of "coins" ($50, $20 $10 $5 $2 and $1) one would receive after completing the transaction:
(define (change total payment)
(let [(x (- payment total))]
(change-aux x '(50 20 10 5 2 1))))
(define (change-aux change coins)
(cond
[(empty? coins) empty]
[else (let [(num-coins (floor (/ change (car coins))))]
(append (list num-coins)
(change-aux (- change (* (car coins) num-coins)) (cdr coins))))]))
So, if I input these parameters:
> (change 44 200)
It returns the output:
'(3 0 0 1 0 1)
That's 200-44 = 156, which corresponds to 3 coins worth $50, 1 worth $5 and 1 worth $1.
My question would be if there's a more elegant, simplified way to write a similar procedure without relying on auxiliary functions, and rather use lambda, filter, map, foldr, foldl etc?
Thanks in advance.
Here is a solution in a different Lisp dialect which shows how to do it with a left fold (reduce) without any mutation of an accumulator variable, as a kind of functional counterpoint to the existing solution.
(defun change (amount coins)
(reduce-left (tb ((counts . rem) next-coin)
(let* ((coin-count (floor rem next-coin))
(coin-value (* coin-count next-coin)))
(cons (cons coin-count counts)
(- rem coin-value))))
coins
(cons '() amount)))
3> (change 156 '(50 20 10 5 2 1))
((1 0 1 0 0 3) . 0)
4> (change 88 '(50 20 10 5 2 1))
((1 1 1 1 1 1) . 0)
Note that the values end up reported in reverse order and wrapped in an extra cons cell; a "porcelain" function could be used around this "plumbing" to report the result in the expected form.
The idea is that we have an accumulator which looks like this: (counts . remainder). The counts part of the accumulator stored in the car is the list of coins accumulated so far. When the reduce is done, this holds the final list. The cdr field holds the remaining amount to be processed; since the last coin is 1, this will always emerge as zero.
Using this accumulator structure, we process the list of coins.
On each call to our reduce kernel function, the left argument is the accumulator, and the right argument, next-coin, is the next coin denomination value.
I used a macro called tb ("tree bind") macro, which is a kind of lambda that provides built-in destructuring, to make it look like we have three parameters.
The initial value for the reduce job is the starting accumulator, which has an empty list of coins, and the full original amount: (cons nil amount) (which I rewrote to (cons '() amount) for better Scheme compatibility).
The reduce function is very simple: greedily calculate how many of the next coin value are needed to represent the remainder, and then calculate the new remainder, packaging these up into a new accumulator cons cell that is returned, and is passed to the next invocation of the function, or returned when the list of coin values has been processed.
Hopefully this points the way to "a more elegant, simplified way to write a similar procedure without relying on auxiliary functions, and rather use lambda, filter, map, foldr, foldl etc" that you can work out in Racket. Good luck!
Sure, you can.
Final solution
(define (change total payment (coins '(50 20 10 5 2 1)))
(let ((rest-diff (- payment total)))
(map (lambda (coin)
(let ((num-coins (floor (/ rest-diff coin))))
(set! rest-diff (- rest-diff (* num-coins coin)))
num-coins))
coins)))
Step by step
First of all, using inner define, you can get rid of the auxiliary function from the global namespace.
(define (change total payment)
(define (change-aux change coins)
(cond
[(empty? coins) empty]
[else (let [(num-coins (floor (/ change (car coins))))]
(append (list num-coins)
(change-aux (- change (* (car coins) num-coins)) (cdr coins))))]))
(let [(x (- payment total))]
(change-aux x '(50 20 10 5 2 1))))
Then, you can pull some variables of the helper function's to the global function's lambda list.
(define (change total payment (coins '(50 20 10 5 2 1)))
(define (change-aux change) ;; eliminate coins in the inner lambda list
(cond
[(empty? coins) empty] ;; coins in function body looked up from outer arguments
[else (let [(num-coins (floor (/ change (car coins))))]
(append (list num-coins)
(change-aux (- change (* (car coins) num-coins)) (cdr coins))))]))
(let [(x (- payment total))]
(change-aux x))) ;; eliminate coins in the call
Then, looking at the code of change-aux, one understands this is actually
a looping through and trying to fit maximal multiples of current value
into the rest of the difference remaining - and collecting those reults. One could loop using map and use set! to mutate the rest.
(define (change total payment (coins '(50 20 10 5 2 1)))
(let ((rest-diff (- payment total)))
(map (lambda (coin)
(let ((num-coins (floor (/ rest-diff coin))))
(set! rest-diff (- rest-diff (* num-coins coin)))
num-coins))
coins)))
Then, you call like above:
(change 44 200)
;; '(3 0 0 1 0 1)

Function that is always evaluated to t (or any value)

Say, we would like to use a function that requires a predicate, but for some reason we're interested in other features of the function (like :start and :end parameters), so we need to supply a predicate, that always returns true.
Obviously, it's not a problem at all:
CL-USER> (defparameter *list* '(0 1 2 3 4 5))
*LIST*
CL-USER> (remove-if (lambda (x) t) *list* :start 1 :end 3)
(0 3 4 5)
Works, but not beautifully at all. And we may get ugly message that variable x is not used. Since I'm in LISP for beauty, I'm curious if there is a 'always t' predicate?
We can define it:
(defun tp (&rest rest)
(declare (ignore rest))
t)
..but may be it exists?
You're looking for the function constantly that takes an argument and returns a function that always returns that value. The predicate you need, then is (constantly t). Thus:
CL-USER> (remove-if (constantly t) '(0 1 2 3 4 5) :start 1 :end 3)
(0 3 4 5)
The notes on constantly show that you were absolutely on the right track with your proposed implementation. (You did even better, though, by adding the (declare (ignore …)).)
Notes:
constantly could be defined by:
(defun constantly (object)
#'(lambda (&rest arguments) object))
After writing this, it occurred to me that this might be a duplicate. I didn't find a proper duplicate, found a similar, more specific, question that was looking to remove a single element at a position, Is there a common lisp macro for popping the nth element from a list?, in which Rainer Joswig's answer includes:
Removing the nth element of a list:
(defun remove-nth (list n)
(remove-if (constantly t) list :start n :end (1+ n)))
This is really just a generalization of that approach, since you're working with arbitrary sequence boundaries. Thus we could have (making the boundary argument analogous to subseq's):
(defun remove-subseq (sequence &optional (start 0) end)
(remove-if (constantly t) sequence :start start :end end))
(defun remove-nth (sequence n)
(remove-subseq sequence n (1+ n)))
CL-USER> (remove-subseq '(0 1 2 3 4 5) 1 3)
(0 3 4 5)
CL-USER> (remove-nth '(0 1 2 3 4 5) 3)
(0 1 2 4 5)

How to sum all numbers that are divisible by 3 or 5 below 1000 in Lisp?

I started programming in Common Lisp yesterday. Now I want to find the sum of all the multiples of 3 or 5 below 1000. I came up with:
(loop for n from 1 to 1000 when
(or
(eq (mod n 5) 0)
(eq (mod n 3) 0))
(sum n)))
I know that the looping part works (loop for n from 1 to 1000 sum n) to sum the first 1000 numbers. I know that the ((eq (mod n 5) 0) (mod n 3) 0)) parts works. And I know that (or (eq (mod n 5) 0) (eq (mod n 3) 0)) works. So it looks like a robust program to me, but when I run it I get the error:
1=(SUM N) found where keyword expected getting LOOP clause after WHEN current LOOP context: WHEN (OR (EQ (MOD 1000 5) 0)
(EQ (MOD 1000 3) 0))
1#. [Condition of type SB-INT:SIMPLE-PROGRAM-ERROR]
I suspect something is wrong with the (sum n) after the or-statement. But I do not know why that is or how I can solve it. Can someone help me out and get my first Lisp program to work?
sum n, not (sum n)
Don't put sum n in parentheses. The loop macro is its own domain specific language with its own grammar. With it, you'd (loop for ... sum n). The grammar is given in HyperSpec entry on loop in this production:
numeric-accumulation::= {count | counting | sum | summing | }
maximize | maximizing | minimize | minimizing {form | it}
[into simple-var] [type-spec]
If it sounds any better to, you can also write (loop for … summing n). That might read more like a natural English sentence.
=, eql, or zerop, but not eq
It's good practice to get into looking up functions, macros, etc., in the HyperSpec. As Rainer Joswig points out, you shouldn't use eq for comparing numbers. Why? Let's look it up in the HyperSpec. The examples include:
(eq 3 3)
=> true
OR=> false
(eq 3 3.0) => false
(eq 3.0 3.0)
=> true
OR=> false
(eq #c(3 -4) #c(3 -4))
=> true
OR=> false
and the Notes section says (emphasis added):
Objects that appear the same when printed are not necessarily eq to
each other. Symbols that print the same usually are eq to each other
because of the use of the intern function. However, numbers with the
same value need not be eq, and two similar lists are usually not
identical.
An implementation is permitted to make "copies" of characters and
numbers at any time. The effect is that Common Lisp makes no guarantee
that eq is true even when both its arguments are "the same thing" if
that thing is a character or number.
For numbers you need something else. = is a good general numeric comparison, although it does more work here than what you need, because it can compare numbers of different types. E.g., (= 5 5.0) is true. Since you're only concerned about 0, you could use zerop, but that will still do a bit more work than you need, since it will check other numeric types as well. E.g., (zerop #c(0.0 0.0)) is true. In this case, since (mod n …) will be giving you an integer, you can use eql:
The value of eql is true of two objects, x and y, in the folowing
cases:
If x and y are eq.
If x and y are both numbers of the same type and the same value.
If they are both characters that represent the same character.
Thus, you can use (or (eql (mod n 3) 0) (eql (mod n 5) 0)).
Other ways of doing this
Now, your question was about a particular piece of loop syntax, and there were some points to be made about equality operators. However, since some of the other answers have looked at other ways to do this, I think it's worth pointing out that there are much more efficient ways to do this. First, let's look at a way to sum up all the multiples of some number beneath a given limit. E.g., for the number 3 and the inclusive limit 26, we have the sum
?
= 3 + 6 + 9 + 12 + 15 + 18 + 21 + 24
= (3 + 24) + (6 + 21) + (9 + 18) + (12 + 15)
= 27 + 27 + 27 + 27
In general, if you try with a few different numbers, you can work out that for an inclusive limit l and a number n, you'll be adding up pairs of numbers, with an optional half pair if there's a odd number of multiples of n that are less than l. I'm not going to work out the whole derivation, but you can end up with
(defun sum-of-multiples-below-inclusive (limit divisor)
(multiple-value-bind (quotient remainder)
(floor limit divisor)
(let ((pair (+ (- limit remainder) divisor)))
(multiple-value-bind (npairs half-pair)
(floor quotient 2)
(+ (* npairs pair)
(if (oddp half-pair)
(floor pair 2)
0))))))
Then, to find out the sum of the number of multiples less than a given number, you can just substract one from limit:
(defun sum-of-multiples-below (limit divisor)
(sum-of-multiples-below (1- limit) divisor))
Then, to expand to your case, where there are multiple divisors, you'll need to add some of these numbers, and then subtract out the ones that are getting counted twice. E.g., in your case:
(+ (sum-of-multiples-below 1000 3)
(sum-of-multiples-below 1000 5)
(- (sum-of-multiples-below 1000 15)))
;=> 233168
(loop for i from 1 below 1000
when (or (eql 0 (mod i 3))
(eql 0 (mod i 5)))
sum i)
;=> 233168
Now, using time naively can lead to misleading results, but SBCL compiles forms before it evaluates them, so this isn't too terrible. This is a very, very, small micro-benchmark, but take a look at the number of cycles used in each form:
(time (+ (sum-of-multiples-below 1000 3)
(sum-of-multiples-below 1000 5)
(- (sum-of-multiples-below 1000 15))))
Evaluation took:
0.000 seconds of real time
0.000000 seconds of total run time (0.000000 user, 0.000000 system)
100.00% CPU
11,327 processor cycles
0 bytes consed
(time (loop for i from 1 below 1000
when (or (eql 0 (mod i 3))
(eql 0 (mod i 5)))
sum i))
Evaluation took:
0.000 seconds of real time
0.000000 seconds of total run time (0.000000 user, 0.000000 system)
100.00% CPU
183,843 processor cycles
0 bytes consed
Using the closed form is much faster. The different is more pronounced if we use a higher limit. Let's look at 100,000:
(time (+ (sum-of-multiples-below 100000 3)
(sum-of-multiples-below 100000 5)
(- (sum-of-multiples-below 100000 15))))
Evaluation took:
0.000 seconds of real time
0.000000 seconds of total run time (0.000000 user, 0.000000 system)
100.00% CPU
13,378 processor cycles
0 bytes consed
(time (loop for i from 1 below 100000
when (or (eql 0 (mod i 3))
(eql 0 (mod i 5)))
sum i))
Evaluation took:
0.007 seconds of real time
0.004000 seconds of total run time (0.004000 user, 0.000000 system)
57.14% CPU
18,641,205 processor cycles
0 bytes consed
For 10,000,000 the numbers are even more staggering:
(time (+ (sum-of-multiples-below 10000000 3)
(sum-of-multiples-below 10000000 5)
(- (sum-of-multiples-below 10000000 15))))
Evaluation took:
0.000 seconds of real time
0.000000 seconds of total run time (0.000000 user, 0.000000 system)
100.00% CPU
13,797 processor cycles
0 bytes consed
(time (loop for i from 1 below 10000000
when (or (eql 0 (mod i 3))
(eql 0 (mod i 5)))
sum i))
Evaluation took:
0.712 seconds of real time
0.712044 seconds of total run time (0.712044 user, 0.000000 system)
100.00% CPU
1,916,513,379 processor cycles
0 bytes consed
Some of these Project Euler problems are pretty interesting. Some of them have some pretty straightforward naïve solutions that work for small inputs, but don't scale well at all.
I would format the code like this:
(loop for n from 1 below 1000
when (or (zerop (mod n 3))
(zerop (mod n 5)))
sum n))
one line less
when at the start of a line
no need to have or alone on a line
clauses in LOOP don't have parentheses
use below
This kind of Loop macro goes back to the early 70s to Interlisp, long before Common Lisp existed.
Here is a another solution without loop
(defun sum-to-thousand (count result)
(cond ((> count 1000) result)
((= (mod count 3) 0) (sum-to-thousand (+ count 1) (+ count result)))
((= (mod count 5) 0) (sum-to-thousand (+ count 1) (+ count result)))
(t (sum-to-thousand (+ count 1) result))))
May I propose more "lispier" variant:
CL-USER> (defun my-sum (&key (from 1) to dividers (sum 0))
(if (>= from to)
sum
(my-sum :from (1+ from)
:to to
:dividers dividers
:sum (if (some (lambda (x) (zerop (mod from x))) dividers)
(+ sum from)
sum))))
MY-SUM
CL-USER> (my-sum :to 1000 :dividers '(3 5))
233168

Common Lisp: "no non-white-space characters in string"

For Project Euler Problem 8, I am told to parse through a 1000 digit number.
This is a brute-force Lisp solution, which basically goes through every 5 consecutive digits and multiplies them from start to finish, and returns the largest one at the end of the loop.
The code:
(defun pep8 ()
(labels ((product-of-5n (n)
(eval (append '(*)
(loop for x from n to (+ n 5)
collect (parse-integer
1000digits-str :start x :end (+ x 1)))))))
(let ((largestproduct 0))
(do ((currentdigit 0 (1+ currentdigit)))
((> currentdigit (- (length 1000digits-str) 6)) (return largestproduct))
(when (> (product-of-5n currentdigit) largestproduct)
(setf largestproduct (product-of-5n currentdigit)))))))
It compiles without any warnings, but upon running it I get:
no non-whitespace characters in string "73167176531330624919225119674426574742355349194934...".
[Condition of type SB-INT:SIMPLE-PARSE-ERROR]
I checked to see if the local function product-of-5n was working by writing it again as a global function:
(defun product-of-5n (n)
(eval (append '(*)
(loop for x from n to (+ n 5)
collect (parse-integer
1000digits-str :start x :end (+ x 1))))))
This compiled without warnings and upon running it, appears to operate perfectly. For example,
CL_USER> (product-of-5n 1) => 882
Which appears to be correct since the first five digits are 7, 3, 1, 6 and 7.
As for 1000digits-str, it was simply compiled with defvar, and with Emacs' longlines-show-hard-newlines, I don't think there are any white-space characters in the string, because that's what SBCL is complaining about, right?
I don't think there are any white-space characters in the string, because that's what SBCL is complaining about, right?
The error-message isn't complaining about the presence of white-space, but about the absence of non-white-space. But it's actually a bit misleading: what the message should say is that there's no non-white-space in the specific substring to be parsed. This is because you ran off the end of the string, so were parsing a zero-length substring.
Also, product-of-5n is not defined quite right. It's just happenstance that (product-of-5n 1) returns the product of the first five digits. Strings are indexed from 0, so (product-of-5n 1) starts with the second character; and the function iterates from n + 0 to n + 5, which is a total of six characters; so (product-of-5n 1) returns 3 × 1 × 6 × 7 × 1 × 7, which happens to be the same as 7 × 3 × 1 × 6 × 7 × 1.
EVAL is not a good idea.
Your loop upper bound is wrong.
Otherwise I tried it with the number string and it works.
It's also Euler 8, not 9.
This is my version:
(defun euler8 (string)
(loop for (a b c d e) on (map 'list #'digit-char-p string)
while e maximize (* a b c d e)))
since I don't know common lisp, I slightly modified your code to fit with elisp. As far as finding bugs go and besides what have been said ((product-of-5n 1) should return 126), the only comment I have is that in (pep8), do length-4 instead of -6 (otherwise you loose last 2 characters). Sorry that I don't know how to fix your parse-error (I used string-to-number instead), but here is the code in case you find it useful:
(defun product-of-5n (n) ;take 5 characters from a string "1000digits-str" starting with nth one and output their product
(let (ox) ;define ox as a local variable
(eval ;evaluate
(append '(*) ;concatenate the multiplication sign to the list of 5 numbers (that are added next)
(dotimes (x 5 ox) ;x goes from 0 to 4 (n is added later to make it go n to n+4), the output is stored in ox
(setq ox (cons ;create a list of 5 numbers and store it in ox
(string-to-number
(substring 1000digits-str (+ x n) (+ (+ x n) 1) ) ;get the (n+x)th character
) ;end convert char to number
ox ) ;end cons
) ;end setq
) ;end dotimes, returns ox outside of do, ox has the list of 5 numbers in it
) ;end append
) ;end eval
) ;end let
)
(defun pep8 () ;print the highest
(let ((currentdigit 0) (largestproduct 0)) ;initialize local variables
(while (< currentdigit (- (length 1000digits-str) 4) ) ;while currentdigit (cd from now on) is less than l(str)-4
;(print (cons "current digit" currentdigit)) ;uncomment to print cd
(when (> (product-of-5n currentdigit) largestproduct) ;when current product is greater than previous largestproduct (lp)
(setq largestproduct (product-of-5n currentdigit)) ;save lp
(print (cons "next good cd" currentdigit)) ;print cd
(print (cons "with corresponding lp" largestproduct)) ;print lp
) ;end when
(setq currentdigit (1+ currentdigit)) ;increment cd
) ;end while
(print (cons "best ever lp" largestproduct) ) ;print best ever lp
) ;end let
)
(setq 1000digits-str "73167176531330624919")
(product-of-5n 1)
(pep9)
which returns (when ran on the first 20 characters)
"73167176531330624919"
126
("next good cd" . 0)
("with corresponding lp" . 882)
("next good cd" . 3)
("with corresponding lp" . 1764)
("best ever lp" . 1764)
I've done this problem some time ago, and there's one thing you are missing in the description of the problem. You need to read consequent as starting at any offset into a sting, not only the offsets divisible by 5. Therefore the solution to the problem will be more like the following:
(defun pe-8 ()
(do ((input (remove #\Newline
"73167176531330624919225119674426574742355349194934
96983520312774506326239578318016984801869478851843
85861560789112949495459501737958331952853208805511
12540698747158523863050715693290963295227443043557
66896648950445244523161731856403098711121722383113
62229893423380308135336276614282806444486645238749
30358907296290491560440772390713810515859307960866
70172427121883998797908792274921901699720888093776
65727333001053367881220235421809751254540594752243
52584907711670556013604839586446706324415722155397
53697817977846174064955149290862569321978468622482
83972241375657056057490261407972968652414535100474
82166370484403199890008895243450658541227588666881
16427171479924442928230863465674813919123162824586
17866458359124566529476545682848912883142607690042
24219022671055626321111109370544217506941658960408
07198403850962455444362981230987879927244284909188
84580156166097919133875499200524063689912560717606
05886116467109405077541002256983155200055935729725
71636269561882670428252483600823257530420752963450"))
(tries 0 (1+ tries))
(result 0))
((= tries 5) result)
(setq result
(max result
(do ((max 0)
(i 0 (+ 5 i)))
((= i (length input)) max)
(setq max
(do ((j i (1+ j))
(current 1)
int-char)
((= j (+ 5 i)) (max current max))
(setq int-char (- (char-code (aref input j)) 48))
(case int-char
(0 (return max))
(1)
(t (setq current (* current int-char))))))))
input (concatenate 'string (subseq input 1) (subseq input 0 1)))))
It's a tad ugly, but it illustrates the idea.
EDIT sorry, I've confused two of your functions. So that like was incorrect.

Is it correct to use the backtick / comma idiom inside a (loop ...)?

I have some code which collects points (consed integers) from a loop which looks something like this:
(loop
for x from 1 to 100
for y from 100 downto 1
collect `(,x . ,y))
My question is, is it correct to use `(,x . ,y) in this situation?
Edit: This sample is not about generating a table of 100x100 items, the code here just illustrate the use of two loop variables and the consing of their values. I have edited the loop to make this clear. The actual loop I use depends on several other functions (and is part of one itself) so it made more sense to replace the calls with literal integers and to pull the loop out of the function.
It would be much 'better' to just do (cons x y).
But to answer the question, there is nothing wrong with doing that :) (except making it a tad slower).
I think the answer here is resource utilization (following from This post)
for example in clisp:
[1]> (time
(progn
(loop
for x from 1 to 100000
for y from 1 to 100000 do
collect (cons x y))
()))
WARNING: LOOP: missing forms after DO: permitted by CLtL2, forbidden by ANSI
CL.
Real time: 0.469 sec.
Run time: 0.468 sec.
Space: 1609084 Bytes
GC: 1, GC time: 0.015 sec.
NIL
[2]> (time
(progn
(loop
for x from 1 to 100000
for y from 1 to 100000 do
collect `(,x . ,y)) ;`
()))
WARNING: LOOP: missing forms after DO: permitted by CLtL2, forbidden by ANSI
CL.
Real time: 0.969 sec.
Run time: 0.969 sec.
Space: 10409084 Bytes
GC: 15, GC time: 0.172 sec.
NIL
[3]>
dsm: there are a couple of odd things about your code here. Note that
(loop for x from 1 to 100000
for y from 1 to 100000 do
collect `(,x . ,y))
is equivalent to:
(loop for x from 1 to 100
collecting (cons x x))
which probably isn't quite what you intended. Note three things: First, the way you've written it, x and y have the same role. You probably meant to nest loops. Second, your do after the y is incorrect, as there is not lisp form following it. Thirdly, you're right that you could use the backtick approach here but it makes your code harder to read and not idiomatic for no gain, so best avoided.
Guessing at what you actually intended, you might do something like this (using loop):
(loop for x from 1 to 100 appending
(loop for y from 1 to 100 collecting (cons x y)))
If you don't like the loop macro (like Kyle), you can use another iteration construct like
(let ((list nil))
(dotimes (n 100) ;; 0 based count, you will have to add 1 to get 1 .. 100
(dotimes (m 100)
(push (cons n m) list)))
(nreverse list))
If you find yourself doing this sort of thing a lot, you should probably write a more general function for crossing lists, then pass it these lists of integers
If you really have a problem with iteration, not just loop, you can do this sort of thing recursively (but note, this isn't scheme, your implementation may not guaranteed TCO). The function "genint" shown by Kyle here is a variant of a common (but not standard) function iota. However, appending to the list is a bad idea. An equivalent implementation like this:
(defun iota (n &optional (start 0))
(let ((end (+ n start)))
(labels ((next (n)
(when (< n end)
(cons n (next (1+ n))))))
(next start))))
should be much more efficient, but still is not a tail call. Note I've set this up for the more usual 0-based, but given you an optional parameter to start at 1 or any other integer. Of course the above can be written something like:
(defun iota (n &optional (start 0))
(loop repeat n
for i from start collecting i))
Which has the advantage of not blowing the stack for large arguments. If your implementation supports tail call elimination, you can also avoid the recursion running out of place by doing something like this:
(defun iota (n &optional (start 0))
(labels ((next (i list)
(if (>= i (+ n start))
nil
(next (1+ i) (cons i list)))))
(next start nil)))
Hope that helps!
Why not just
(cons x y)
By the way, I tried to run your code in CLISP and it didn't work as expected. Since I'm not a big fan of the loop macro here's how you might accomplish the same thing recursively:
(defun genint (stop)
(if (= stop 1) '(1)
(append (genint (- stop 1)) (list stop))))
(defun genpairs (x y)
(let ((row (mapcar #'(lambda (y)
(cons x y))
(genint y))))
(if (= x 0) row
(append (genpairs (- x 1) y)
row))))
(genpairs 100 100)