Bignum overflow error after Euler #2 attempt - lisp

I've attempted to solve Euler Problem 2 with the following tail recursive functions:
(defun fib (num)
(labels ((fib-helper (num a b)
(cond ((or (zerop num)
(eql num 1))
a)
(t (fib-helper (decf num)
(+ a b)
a)))))
(fib-helper num 1 1)))
(defun sum-even-fib (max)
(labels ((helper (sum num)
(cond ((oddp num) (helper sum (decf num)))
((zerop num) sum)
(t (helper (+ sum (fib num))
(decf num))))))
(helper 0 max)))
Now, when I try to print the result using the function
(defun print-fib-sum (max dir file)
(with-open-file
(fib-sum-str
(make-pathname
:name file
:directory dir)
:direction :output)
(format fib-sum-str "~A~%" (sum-even-fib max))))
with a max value of 4000000, I get the error
("bignum overflow" "[Condition of type SYSTEM::SIMPLE-ARITHMETIC-ERROR]" nil)
from *slime-events*. Is there any other way to handle the number and print to the file?

First, a few small issues:
Use time instead of top.
Common Lisp standard does not require tail recursion optimization. While many implementation do it, not all of them optimize all cases (e.g., labels).
Your algorithm is quadratic in max because it computes the nth Fibonacci number separately for all indexes. You should make it linear instead.
You are computing the sum of even-indexed numbers, not even-valued numbers.
Now, the arithmetic error you are seeing: 4,000,000th Fibonacci number is pretty large - about 1.6^4M ~ 10^835951. Its length is about 2,776,968.
Are you sure your lisp can represent bignums this big?

So I've solved Euler #2 with the following tail recursive code:
(defun rev-sum-even-fib (max-val)
(labels ((helper (sum a b)
(cond ((oddp a)
(helper sum (+ a b) a))
((> a max-val)
sum)
(t
(helper (+ sum a) (+ a b) a)))))
(helper 0 1 0)))
Here, the algorithm is linear in max and evaluates in
(time (rev-sum-even-fib 4000000))
Real time: 3.4E-5 sec.
Run time: 0.0 sec.
Space: 0 Bytes
Where I've omitted the numerical answer for obvious reasons.

Since CL does not promise that it supports TCO (for example ABCL on the JVM does not support TCO - tail call optimization), it makes sense to write it portably as a loop:
(defun rev-sum-even-fib (max-val)
(loop for a = 1 then (+ a b) and b = 0 then a
until (> a max-val)
when (evenp a) sum a))

Related

LISP: how to get running sum of a list? (without a global variable)

I am a LISP newbie.
To get the running sum of a list, I am writing like --
(setf sum 0.0)
(mapcar #'(lambda(x)
(setf sum (+ sum x)) sum) values))
For example, if you give '(1 2 3 4) as input, the above code returns '(1 3 6 10) as output and so forth.
Is it possible to do the same thing (in a more elegant way) without using the global variable sum ?
(loop for x in '(1 2 3 4) sum x into y collect y)
scanl is a oneliner:
(defun scanl (f init xs)
(loop for x in xs collect (setf init (funcall f init x))))
You could use loop, like this:
(defun running-sum (xs)
(loop with sum = 0
for x in xs
collect (setf sum (+ sum x))))
(running-sum '(1 2 3 4))
It's fundamentally the same thing, but it uses a local variable instead of a global one, and might be more clear.
Alternatively, you could define a recursive function, and a wrapper function:
(defun running-sum-recursive (xs)
(running-sum-recursive2 0 xs))
(defun running-sum-recursive2 (sum xs)
(if (eq xs nil)
nil
(let ((new-sum (+ sum (car xs))))
(cons new-sum (running-sum-recursive2 new-sum (cdr xs))))))
(running-sum-recursive '(1 2 3 4))
However this seems needlessly complicated to me when loop is available.
Note that in Haskell, you could do a running sum like this:
runningSum xs = scanl1 (+) xs
runningSum [1, 2, 3, 4]
The key here is the scanl1 function. It's possible that something similar exists in Lisp (and we've very nearly written it twice now), but I haven't used Lisp in a while.
Edit: After some searching, I don't think Common Lisp includes anything quite like scanl or scanl1, so here they are:
(defun scanl (f val xs)
(loop for x in xs
collect (setf val (funcall f val x))))
(defun scanl1 (f xs)
(cons (car xs)
(scanl f (car xs) (cdr xs))))
(scanl1 #'+ '(1 2 3 4))
Edit: Thanks to huaiyuan's answer for a suggestion about how the loops could be shortened.
Or you could use higher-order functions
(define (running-sum ls)
(cdr (reverse (foldl (lambda (y xs) (cons (+ (car xs) y) xs)) '(0) ls))))
Haskell does have a rich inventory of functions for list recursion, but we've got reduce at least. Here is an elementary (i. e. without the loop magic) functional solution:
(defun running-sum (lst)
(reverse (reduce (lambda (acc x)
(cons (+ (first acc) x) acc))
(rest lst)
:initial-value (list (first lst)))))
I'm using the head of the original list as the initial value and walk through the rest of the list adding sums at the head (because it's natural to add at the head), finally reversing the list thus obtained.
One can use reduce in most cases when there's a need to traverse a sequence accumulating a value.
Here is an elementary iterative solution using the push-nreverse idiom:
(defun running-sum (lst)
(let ((sums (list (first lst))))
(dolist (x (rest lst))
(push (+ x (first sums)) sums))
(nreverse sums)))
In Scheme I would calculate the sum of the list recursively using an accumulator. Like so:
; Computes a list of intermediary results of list summation
(define list-sum
(lambda (l)
(letrec ((recsum (lambda (lst acc acclst)
(if (pair? lst)
(recsum (cdr lst) (+ acc (car lst)) (cons acc acclst))
(cons acc acclst)))))
(recsum (cdr l) (car l) '()))))
Output:
> (list-sum '(1 2 3 4))
(10 6 3 1)
> (list-sum '(2 4 6 8 10))
(30 20 12 6 2)
>
The trick to recurse over a list is to take the first element/car off each time and pass the rest/cdr. You can keep intermediary results by using an extra parameter (called an accumulator) and pass the sum in that. I've used two accumulators above: one for the last sum and one for a list of all previous sums.
I've never done anything in LISP, so I can't tell if this translates directly to your dialect(?), but it's conceptually simple and I'm sure it's doable in LISP as well.
Do ask if something is not immediately clear. It's been a while since I've used this family of languages :)

Feedback on lisp program for project euler 4 [closed]

Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 10 years ago.
Improve this question
I just started learning common lisp and so I've been working on project euler problems. Here's my solution (with some help from https://github.com/qlkzy/project-euler-cl ). Do you guys have any suggestions for stylistic changes and the sort to make it more lisp-y?
; A palindromic number reads the same both ways. The largest palindrome made
; from the product of two 2-digit numbers is 9009 = 91 99.
; Find the largest palindrome made from the product of two 3-digit numbers.
(defun num-to-list (num)
(let ((result nil))
(do ((x num (truncate x 10)))
((= x 0 ) result)
(setq result (cons (mod x 10) result)))))
(defun palindrome? (num)
(let ((x (num-to-list num)))
(equal x (reverse x))))
(defun all-n-digit-nums (n)
(loop for i from (expt 10 (1- n)) to (1- (expt 10 n)) collect i))
(defun all-products-of-n-digit-nums (n)
(let ((nums (all-n-digit-nums n)))
(loop for x in nums
appending (loop for y in nums collecting (* x y)))))
(defun all-palindromes (n)
(let ((nums (all-products-of-n-digit-nums n)))
(loop for x in nums
when (palindrome? x) collecting x)))
(defun largest-palindrome (n)
(apply 'max (all-palindromes 3)))
(print (largest-palindrome 3))
Barnar's solution is great however there's just a small typo, to return a result it should be:
(defun largest-palindrome (n)
(loop with start = (expt 10 (1- n))
and end = (1- (expt 10 n))
for i from start to end
maximize (loop for j from i to end
for num = (* i j)
when (palindrome? num)
maximize num)))
(setq list (cons thing list))
can be simplified to:
(push thing list)
My other comments on your code are not so much about Lisp style as about the algorithm. Creating all those intermediate lists of numbers seems like a poor way to do it, just write nested loops that calculate and test the numbers.
(defun all-palindromes (n)
(loop for i from (expt 10 (1- n)) to (1- (expt 10 n))
do (loop for j from (expt 10 (1- n)) to (1- (expt 10 n))
for num = (* i j)
when (palindrome? num)
collect num)))
But LOOP has a feature you can use: MAXIMIZE. So instead of collecting all the palindroms in a list with COLLECT, you can:
(defun largest-palindrome (n)
(loop with start = (expt 10 (1- n))
and end = (1- (expt 10 n))
for i from start to end
do (loop for j from start to end
for num = (* i j)
when (palindrome? num)
maximize num)))
Here's another optimization:
(defun largest-palindrome (n)
(loop with start = (expt 10 (1- n))
and end = (1- (expt 10 n))
for i from start to end
do (loop for j from i to end
for num = (* i j)
when (palindrome? num)
maximize num)))
Making the inner loop start from i instead of start avoids the redundancy of checking both M*N and N*M.
The example below is a bit contrived, but it finds the palindrome in a lot less iterations than your original approach:
(defun number-to-list (n)
(loop with i = n
with result = nil
while (> i 0) do
(multiple-value-bind (a b)
(floor i 10)
(setf i a result (cons b result)))
finally (return result)))
(defun palindrome-p (n)
(loop with source = (coerce n 'vector)
for i from 0 below (floor (length source) 2) do
(when (/= (aref source i) (aref source (- (length source) i 1)))
(return))
finally (return t)))
(defun suficiently-large-palindrome-of-3 ()
;; This is a fast way to find some sufficiently large palindrome
;; that fits our requirement, but may not be the largest
(loop with left = 999
with right = 999
for maybe-palindrome = (number-to-list (* left right)) do
(cond
((palindrome-p maybe-palindrome)
(return (values left right)))
((> left 99)
(decf left))
((> right 99)
(setf left 999 right (1- right)))
(t ; unrealistic situation
; we didn't find any palindromes
; which are multiples of two 3-digit
; numbers
(return)))))
(defun largest-palindrome-of-3 ()
(multiple-value-bind (left right)
(suficiently-large-palindrome-of-3)
(loop with largest = (* left right)
for i from right downto left do
(loop for j from 100 to 999
for maybe-larger = (* i j) do
(when (and (> maybe-larger largest)
(palindrome-p (number-to-list maybe-larger)))
(setf largest maybe-larger)))
finally (return largest)))) ; 906609
It also tries to optimize a bit the way you check that number is a palindrome, for an additional memory cost though. It also splits the number into a list using somewhat longer code, but making less divisions (which are somewhat computationally expensive).
The whole idea is based on the concept that the largest palindrome will be somewhere more towards the... largest multipliers, so, by starting off with 99 * 99 you will have a lot of bad matches. Instead, it tries to go from 999 * 999 and first find some palindrome, which looks good, doing so in a "sloppy" way. And then it tries hard to improve upon the initial find.

Lisp style question: memoization (caution: contains the solution for project euler #14)

I am just trying to learn some Lisp, so I am going through project euler problems. I found problem no. 14 interesting (so if you are planning to solve this problems stop reading now, because I pasted my solution at the bottom). With my algorithm it was so slow, but after using memoization (I copied the function from Paul Graham's "on Lisp" book) it was much more faster (around 4 to 8 seconds).
My question is about this bunch of warnings that I got:
Am I doing something wrong? Can I improve my style?
> ;; Loading file
> /euler-lisp/euler-14.lisp
> ... WARNING in COLLATZ-SERIE :
> COLLATZ-SERIE-M is neither declared
> nor bound, it will be treated as if it
> were declared SPECIAL. WARNING in
> COLLATZ-SERIE : COLLATZ-SERIE-M is
> neither declared nor bound, it will be
> treated as if it were declared
> SPECIAL. WARNING in COMPILED-FORM-314
> : COLLATZ-SERIE-M is neither declared
> nor bound, it will be treated as if it
> were declared SPECIAL. (525 837799)
> Real time: 18.821894 sec. Run time:
> 18.029127 sec. Space: 219883968 Bytes GC: 35, GC time: 4.080254 sec. Las
> siguientes variables especiales no han
> sido definidas: COLLATZ-SERIE-M 0
> errores, 0 advertencias ;; Loaded file
This is the code:
(defun collatz (n)
(if (evenp n) (/ n 2) (+ (* 3 n) 1)))
(defun memoize (fn)
(let ((cache (make-hash-table :test #'equal)))
#'(lambda (&rest args)
(multiple-value-bind (val win) (gethash args cache)
(if win
val
(setf (gethash args cache)
(apply fn args)))))))
(defun collatz-serie (n)
(cond ((= n 1) (list 1))
((evenp n) (cons n (funcall collatz-serie-m (/ n 2))))
(t (cons n (funcall collatz-serie-m (+ (* 3 n) 1))))))
(defun collatz-serie-len (n)
(length (collatz-serie n)))
(setq collatz-serie-m (memoize #'collatz-serie))
(defun gen-series-pairs (n)
(loop for i from 1 to n collect
(list (collatz-serie-len i) i)))
(defun euler-14 (&key (n 1000000))
(car (sort (gen-series-pairs n) #'(lambda (x y) (> (car x) (car y))))))
(time (print (euler-14)))
Thanks a lot, and forgive the probable errors, I am just beginning with Lisp.
Br
UPDATE:
i want to share the final code that i wrote. using custom external hash table for memoization and improving the final loop.
(defvar *cache* (make-hash-table :test #'equal))
(defun collatz (n)
(if (evenp n) (/ n 2) (+ (* 3 n) 1)))
(defun collatz-serie (n)
(cond ((= n 1) (list 1))
((evenp n) (cons n (collatz-serie (/ n 2))))
(t (cons n (collatz-serie (+ (* 3 n) 1))))))
(defun collatz-serie-new (n)
(labels ((helper (n len)
(multiple-value-bind (val stored?) (gethash n *cache*)
(if stored?
val
(setf (gethash n *cache*) (cond ((= n 1) len)
((evenp n) (+ len (helper (/ n 2) len)))
(t (+ len (helper (+ (* 3 n) 1) len)))))))))
(helper n 1)))
;; learning how to loop
(defun euler-14 (&key (n 1000000))
(loop with max = 0 and pos = 0
for i from n downto 1
when (> (collatz-serie-new i) max)
do (setf max (collatz-serie-new i)) and do (setf pos i)
finally (return (list max pos))))
It is bad style to setq an unknown name. It is assumed that you mean to create a new global special variable, then set it, but this should be made explicit by introducing these bindings first. You do this at the top level by using defvar (or defparameter or defconstant) instead, and in lexical blocks by using let, do, multiple-value-bind or similar constructs.

(define (average ....)) in Lisp

I'm just playing around with scheme/lisp and was thinking about how I would right my own definition of average. I'm not sure how to do some things that I think are required though.
define a procedure that takes an arbitrary number of arguments
count those arguments
pass the argument list to (+) to sum them together
Does someone have an example of defining average? I don't seem to know enough about LISP to form a web search that gets back the results I'm looking for.
The definition would be a very simple one-liner, but without spoiling it, you should look into:
a "rest" argument -- this (define (foo . xs) ...xs...) defines foo as a function that takes any number of arguments and they're available as a list which will be the value of xs.
length returns the length of a list.
apply takes a function and a list of values and applies the function to these values.
When you get that, you can go for more:
see the foldl function to avoid applying a list on a potentially very big list (this can matter in some implementations where the length of the argument list is limited, but it wouldn't make much difference in Racket).
note that Racket has exact rationals, and you can use exact->inexact to make a more efficient floating-point version.
And the spoilers are:
(define (average . ns) (/ (apply + ns) (length ns)))
Make it require one argument: (define (average n . ns) (/ (apply + n ns) (add1 (length ns))))
Use foldl: (define (average n . ns) (/ (foldl + 0 (cons n ns)) (add1 (length ns))))
Make it use floating point: (define (average n . ns) (/ (foldl + 0.0 (cons n ns)) (add1 (length ns))))
In Common Lisp, it looks like you can do:
(defun average (&rest args)
(when args
(/ (apply #'+ args) (length args))))
although I have no idea if &rest is available on all implementations of Lisp. Reference here.
Putting that code into GNU CLISP results in:
[1]> (defun average (&rest args)
(when args
(/ (apply #'+ args) (length args))))
AVERAGE
[2]> (average 1 2 3 4 5 6)
7/2
which is 3.5 (correct).
Two versions in Common Lisp:
(defun average (items)
(destructuring-bind (l . s)
(reduce (lambda (c a)
(incf (car c))
(incf (cdr c) a)
c)
items
:initial-value (cons 0 0))
(/ s l)))
(defun average (items &aux (s 0) (l 0))
(dolist (i items (/ s l))
(incf s i)
(incf l)))
In Scheme, I prefer using a list instead of the "rest" argument because rest argument makes implementing procedures like the following difficult:
> (define (call-average . ns)
(average ns))
> (call-average 1 2 3) ;; => BANG!
Packing arbitrary number of arguments into a list allows you to perform any list operation on the arguments. You can do more with less syntax and confusion. Here is my Scheme version of average that take 'n' arguments:
(define (average the-list)
(let loop ((count 0) (sum 0) (args the-list))
(if (not (null? args))
(loop (add1 count) (+ sum (car args)) (cdr args))
(/ sum count))))
Here is the same procedure in Common Lisp:
(defun average (the-list)
(let ((count 0) (sum 0))
(dolist (n the-list)
(incf count)
(incf sum n))
(/ sum count)))
In Scheme R5RS:
(define (average . numbers)
(/ (apply + numbers) (length numbers)))

Sum of Squares in Lisp

I need to write a non-recursive version of the function sum-squares and Use a do-loop that is based on the length of the argument list.
Here's how it's done generally:
(defun sum-squares (list) (loop for x in list
for y = (* x x)
summing y into total
finally (return total)))
A do loop solution is even simpler, but not half as elegant:
(defun sum-squares (list)
(let ((sum 0)) (do ((i 0 (1+ i)))
((>= i (length list)))
(setq sum (+ sum (* (nth i list) (nth i list)))))
sum))