This question already has an answer here:
Unusual stack overflow when inserting nodes in binary tree
(1 answer)
Closed 6 years ago.
I'm (still) porting code from Franz Lisp to Common LISP. Now I seem to have pushed the interpreter into a strange corner where it blows up.
[11]> (setq fff (cadr contextstack))
*** - Program stack overflow. RESET
Now how can that cause a stack overflow? Where's the recursion?
I can take the length OK:
[12]> (length contextstack)
79
describe works. This is just the beginning of long "describe" output.
[13]> (describe (cadr contextstack))
(# . #1=#) is a cons.
#<FUNCTION POPPATTERN (NODE) (DECLARE (SYSTEM::IN-DEFUN POPPATTERN)) (BLOCK POPPATTERN (MAPONE # #) (XEPATTERN NODE #))> is an interpreted
function.
Argument list: (NODE)
#1=# is a structure of type ENODE.
Slots:
EROOT =
#1=#S(ENODE :EROOT #1# :EQCLASS #1# :ESUCCESSORS ALLTRUE! :ESLENGTH NIL :EPREDECESSORS NIL :ECONGRUENT NIL :EDEMON NIL :EMERGEDEMON NIL
:EPATTERN
;;; much more
But almost anything else applied to contextstack blows up with the stack overflow message.
The structure ENODE contains links to other ENODE items, and there is circularity which could make some code loop. But cadr? How is that
even possible?
The original code used "hunks", a MacLISP feature from the 1970s. I converted it to use Common LISP structs, and may have broken something.
This is all running interpretive; I haven't compiled anything. Safety level is the default. I tried adding (proclaim '(optimize (safety 3))) to force more checking, but it didn't change anything.
If you want to reproduce this, download the code in
https://github.com/John-Nagle/pasv/tree/master/src/CPC4
start "clisp", do (load 'setup), and then start looking at contextstack. Whatever is breaking is happening early, as the program is initializing.
How you got the error
Chances are, the error is in printing, not cadr. You can verify it by doing
(progn (cadr contextstack) nil)
which should work just fine.
If it still blows up, you should check if cadr raises an exception (and then the error message would contain a circular object and blow up):
(and (consp contextstack)
(consp (cdr contextstack)))
How to avoid the error
CLISP FAQ says:
You will always get a stack overflow when you try to print a circular object (LIST, STRUCTURE-OBJECT, VECTOR etc) and *PRINT-CIRCLE* is NIL. Just set *PRINT-CIRCLE* to T.
Note that this is not CLISP-specific, you should get the same error with any ANSI Common Lisp when *print-circle* is nil.
In your specific case, look closely at the output of (describe (cadr contextstack)):
(# . #1=#) is a cons.
This #= notation is used by the
printer to avoid stack overflows when printing circular objects.
PS. You now owe me 10 zorkmids :-)
Related
When using Emacs, SLIME and Clozure CL I have a minor gripe:
The function signature for aref (I have not yet seen any other instances) is shown only as (aref a).
When I go to source the code in question begins with (defun aref (a &lexpr subs). As far as I know, &lexpr is not a valid CL lambda list keyword. So this indicates that SLIME does not show the correct function signature due to the "weird" keyword.
But when I do the same for svref, say, there is nothing (to me at least) that corroborates the above hypthesis. So maybe SLIME does something, too.
Can anybody point to relevant documentation (I did not find anything relevant in the SLIME manual and in the CCL manual) or does anybody have a workaround/solution?
SVREF does not take a list of array indices, since its first argument is a vector. An array might be multi-dimensional, which explains why there are a variadic number of subscripts. For AREF, the possible sources are:
.../ccl/level-0/l0-array.lisp
#'AREF
.../ccl/compiler/optimizers.lisp
(COMPILER-MACRO AREF)
.../ccl/compiler/nx1.lisp
#'CCL::NX1-AREF
Of those, only the first one has the uncommon &lexpr keyword in its argument list.
An experiment:
CL-USER> (defun foobar (a &lexpr b) (list a b))
;Compiler warnings :
; In FOOBAR: Unused lexical variable &LEXPR
FOOBAR
Let's use the unexported symbol from CCL (auto-completion found it):
CL-USER> (defun foobar (a ccl::&lexpr b) (list a b))
FOOBAR
This times, it works, let's try it:
CL-USER> (foobar 0 1 2 3 4)
(0 17563471524599)
The Evolution of Lisp says (emphasis mine):
MacLisp introduced the LEXPR , which is a type of function that takes
any number of arguments and puts them on the stack;
(see https://www.dreamsongs.com/Files/Hopl2.pdf)
Fix by substitution
You can fix it at the level of Swank, by replacing &lexpr by &rest. You only need to patch ccl.lisp, which provides an implementation for arglist for Clozure:
(defimplementation arglist (fname)
(multiple-value-bind (arglist binding) (let ((*break-on-signals* nil))
(ccl:arglist fname))
(if binding
(substitute '&rest 'ccl::&lexpr arglist)
:not-available)))
More details
In fact, in swank-arglists.lisp you can see that the stuff that is unknown is put in a separate list, called :unknown-junk. To see it in action, do:
CL-USER> (trace swank::decoded-arglist-to-string)
NIL
Then, write (aref and press Space, which triggers a query for the argument list and produces the following trace:
0> Calling (SWANK::DECODED-ARGLIST-TO-STRING #S(SWANK/BACKEND:ARGLIST :PROVIDED-ARGS NIL :REQUIRED-ARGS (CCL::A) :OPTIONAL-ARGS NIL :KEY-P NIL :KEYWORD-ARGS NIL :REST NIL :BODY-P NIL :ALLOW-OTHER-KEYS-P NIL :AUX-ARGS NIL :ANY-P NIL :ANY-ARGS NIL :KNOWN-JUNK NIL :UNKNOWN-JUNK (CCL::&LEXPR CCL::SUBS)) :PRINT-RIGHT-MARGIN 159 :OPERATOR AREF :HIGHLIGHT (0))
<0 SWANK::DECODED-ARGLIST-TO-STRING returned "(aref ===> a <===)"
Notice the :UNKNOWN-JUNK (CCL::&LEXPR CCL::SUBS)) part.
Maybe a better fix is to let Swank learn about &lexpr?
I may be asking for the impossible, but am wondering nonetheless.
Is it possible to obtain an analog of the stack-trace for macros? That is, if one set a break-point inside a certain function, the macro-stack-trace would list all macros (perhaps with their inputs) that were macroexpanded to get to that level in the code.
From what I understand, this is currently impossible, but it may be due to my shallow understanding. Does Allegro or SBCL allow or track this kind of information? It appears that this would be really useful for debugging macros.
Any help or advice is appreciated.
As SBCL is a compiler-only implementation meaning all code is automatically compiled (in contrast to being "interpreted"). Calls to macros are expanded as part of compilation, so the fact that something was a macro call is lost.
(defmacro m (n)
`(/ 10 ,n))
(defun foo (x) (m x))
SBCL:
* (foo 0)
debugger invoked on a DIVISION-BY-ZERO in thread
#<THREAD "main thread" RUNNING {1001E06493}>:
arithmetic error DIVISION-BY-ZERO signalled
Operation was /, operands (10 0).
Type HELP for debugger help, or (SB-EXT:EXIT) to exit from SBCL.
restarts (invokable by number or by possibly-abbreviated name):
0: [ABORT] Exit debugger, returning to top level.
(SB-KERNEL::INTEGER-/-INTEGER 10 0)
0] backtrace
Backtrace for: #<SB-THREAD:THREAD "main thread" RUNNING {1001E06493}>
0: (SB-KERNEL::INTEGER-/-INTEGER 10 0)
1: (FOO 0)
2: (SB-INT:SIMPLE-EVAL-IN-LEXENV (FOO 0) #<NULL-LEXENV>)
3: (EVAL (FOO 0))
4: (INTERACTIVE-EVAL (FOO 0) :EVAL NIL)
[...]
Some implementations, e.g. Allegro CL, support both interpreted as well as compiled code, the first being helpful in debugging, the second giving better performance. (I show here the command-line interactions. Allegro also offers a GUI to set breakpoints that I'm not familiar with.)
cl-user(4): (foo 0)
Error: Attempt to divide 10 by zero.
[condition type: division-by-zero]
Restart actions (select using :continue):
0: Return to Top Level (an "abort" restart).
1: Abort entirely from this (lisp) process.
[1] cl-user(5): :zoom
Evaluation stack:
(error division-by-zero :operation ...)
->(/ 10 0)
(foo 0)
(eval (foo 0))
[...]
The zoom command takes many options to be more verbose, this shows the form (block foo (m x)):
[1] cl-user(6): :zoom :all t
Evaluation stack:
... 4 more newer frames ...
((:runsys "lisp_apply"))
[... sys::funcall-tramp ]
(excl::error-from-code 17 nil ...)
(sys::..runtime-operation "integer_divide" :unknown-args)
(excl::/_2op 10 0)
->(/ 10 0)
[... excl::eval-as-progn ]
(block foo (m x))
(foo 0)
(sys::..runtime-operation "comp_to_interp" 0)
[... excl::%eval ]
(eval (foo 0))
When you (compile 'foo) the macro calls will be expanded away (like for SBCL) and not show up in backtraces anymore (but Allegro's source-level debugging could help).
In general when it comes to defining macros, to help debugging try to expand into function calls and not big bodies of code. E.g. instead of:
(defmacro with-foo ((var-x var-y thing) &body body)
`(let ((,var-x (..derive from ,thing ..))
(,var-y (..derive from ,thing ..)))
,#body))
I would write it like:
(defmacro with-foo ((var-x var-y thing) &body body)
`(call-with-foo (lambda (,var-x ,var-y) ,#body) ,thing))
(defun call-with-foo (func thing)
(let ((x (..derive from thing ..)
(y (..derive from thing ..))
(funcall func x y)))
so it ends up in the stack trace and is easy to redefine.
See this great post by Kent Pitman:
Incidentally, too, back to CL, you should know that when I write these
WITH-xxx macros, I almost always accompany them with a CALL-WITH-xxx
so that I can do either kind of call. But I find I almost never use
the CALL-WITH-xxx even when I was the one to provide it as an option.
The main reason I write them is not to use them but to make
redefinition easier, since I can redefine the CALL-WITH-xxx without
redefining the macro, and so I don't have to recompile the callers if
the definition changes.
Yes, AllegroCl supports tracing and in general debugging of macros. Quite an effort for not sure how much benefit, but Franz tends to do good things to make CL more viable. Pro tip: there is a an option to turn off what I think they call source-level debugging of macros, and you will want to do that if your code makes heavy use of macros or compilation times can get crazy. Just turn it back on when you think you need the source debugging.
This question already has answers here:
setq and defvar in Lisp
(4 answers)
Closed 6 years ago.
I was following a tutorial on lisp and they did the following code
(set 'x 11)
(incf x 10)
and interpreter gave the following error:
; in: INCF X
; (SETQ X #:NEW671)
;
; caught WARNING:
; undefined variable: X
;
; compilation unit finished
; Undefined variable:
; X
; caught 1 WARNING condition
21
what is the proper way to increment x ?
This is indeed how you are meant to increment x, or at least one way of doing so. However it is not how you are meant to bind x. In CL you need to establish a binding for a name before you use it, and you don't do that by just assigning to it. So, for instance, this code (in a fresh CL image) is not legal CL:
(defun bad ()
(setf y 2))
Typically this will cause a compile-time warning and a run-time error, although it may do something else: its behaviour is not defined.
What you have done, in particular, is actually worse than this: you have rammed a value into the symbol-value of x (with set, which does this), and then assumed that something like (incf x) will work, which it is extremely unlikely to do. For instance consider something like this:
(defun worse ()
(let ((x 2))
(set 'x 4)
(incf x)
(values x (symbol-value 'x))))
This is (unlike bad) legal code, but it probably does not do what you want it to do.
Many CL implementations do allow assignment to previously unbound variables at the top-level, because in a conversational environment it is convenient. But the exact meaning of such assignments is outwith the language standard.
CMUCL and its derivatives, including SBCL, have historically been rather more serious about this than other implementations were at the time. I think the reason for this is that the interpreter was a bunch more serious than most others and/or they secretly compiled everything anyway and the compiler picked things up.
A further problem is that CL has slightly awkward semantics for top-level variables: if you go to the effort to establish a toplevel binding in the normal way, with defvar & friends, then you also cause the variable to be special -- dynamically scoped -- and this is a pervasive effect: it makes all bindings of that name special. That is often a quite undesirable consequence. CL, as a language, has no notion of a top-level lexical variable.
What many implementations did, therefore, was to have some kind of informal notion of a top-level binding of something which did not imply a special declaration: if you just said (setf x 3) at the toplevel then this would not contage the entire environment. But then there were all sorts of awkward questions: after doing that, what is the result of (symbol-value 'x) for instance?
Fortunately CL is a powerful language, and it is quite possible to define top-level lexical variables within the language. Here is a very hacky implementation called deflexical. Note that there are better implementations out there (including at least one by me, which I can't find right now): this is not meant to be a bullet-proof solution.
(defmacro deflexical (var &optional value)
;; Define a cheap-and-nasty global lexical variable. In this
;; implementation, global lexicals are not boundp and the global
;; lexical value is not stored in the symbol-value of the symbol.
;;
;; This implementation is *not* properly thought-through and is
;; without question problematic
`(progn
(define-symbol-macro ,var (get ',var 'lexical-value))
(let ((flag (cons nil nil)))
;; assign a value only if there is not one already, like DEFVAR
(when (eq (get ',var 'lexical-value flag) flag)
(setf (get ',var 'lexical-value) ,value))
;; Return the symbol
',var)))
This question already has answers here:
Why do some people use keywords for the clauses in the loop macro?
(3 answers)
Closed 8 years ago.
I just saw the answer of Sylwester to this question, and I thought strange that the loop has colons everywhere.
Usually, I would write
(loop for n below 10 do (princ n) (terpri))
instead of
(loop :for n :below 10 :do (princ n) (terpri))
After some tests, I see that with the first loop, the symbols for, below and do are then part of cl-user (edit : actually not do, only the other two, probably because do is also a macro in the cl package), not with the second. Likewise, a 'X alone will then be part of cl-user, not ':X. The symbol-package function tells me the latter is in the keyword package.
Now, the first loop, without colons, looks much prettier to me, so I would like to know if the preceding remarks are a good reason to use the second one instead. That the symbols become "included" in the current package looks rather inoffensive, but maybe I have overlooked the consequences.
Any idea?
You moslty already answered your own question, the difference is as you described. :some-symbol will be in :KEYWORD package, and 'SOME-SYMBOL will be in your current package CL-USER by default. In loop macro it's just a matter of taste. Some people prefer to use :for notation to get better syntax highlighting in their text editor, for example.
CL-USER 23 > (find-symbol "LOOP" "CL")
LOOP
:EXTERNAL
CL-USER 24 > (find-symbol "FOR" "CL")
NIL
NIL
LOOP is a symbol in the COMMON-LISP package and it is exported. FOR is neither. Thus in every package which does not have a FOR symbol and does not inherit one, one will add such a symbol when writing a LOOP FOR loop.
That's it. Usually that should be no problem...
I am learning Lisp from the book "The Land of Lisp" by Conrad Barski. Now I have hit my first stumbling block, where the author says:
Calling yourself in this way is not only allowed in Lisp, but is often
strongly encouraged
after showing the following example function to count the items in a list:
(defun my-length (list)
(if list
(1+ (my-length (cdr list)))
0))
When I call this function my-length with a list containing a million items, I get a stack overflow error. So either you never expect to have a list that long in Lisp (so maybe my use case is useless) or there is another way to count items in such a long list. Can you maybe shine some light on this? (I am using GNU CLISP on Windows, by the way).
TCO (tail call optimization) in CLISP using the example from Chris Taylor:
[1]> (defun helper (acc list)
(if list
(helper (1+ acc) (cdr list))
acc))
(defun my-length (list)
(helper 0 list))
HELPER
Now compile it:
[2]> (compile 'helper)
MY-LENGTH
[3]> (my-length (loop repeat 100000 collect t))
*** - Program stack overflow. RESET
Now, above does not work. Let's set the debug level low. This allows the compiler to do TCO.
[4]> (proclaim '(optimize (debug 1)))
NIL
Compile again.
[5]> (compile 'helper)
HELPER ;
NIL ;
NIL
[6]> (my-length (loop repeat 100000 collect t))
100000
[7]>
Works.
Allowing the Common Lisp compiler to do TCO is most often controlled by the debug level. With a high debug level, the compiler generates code which uses a stack frame for each function call. This way each call can be traced and will be seen in a backtrace. With a lower debug level the compiler may replace tail calls with jumps in the compiled code. These calls then will not be seen in a backtrace - which usually makes debugging harder.
You're looking for tail recursion. At the moment your function is defined like
(defun my-length (list)
(if list
(1+ (my-length (cdr list)))
0))
Notice that after my-length has called itself, it needs to add one to the result before passing that value to its calling function. This need to modify the value before returning it means that you need to allocate a new stack frame for every call, the the space used is proportional to the length of the list. This is what causes a stack overflow on long lists.
You can re-write it to use a helper function
(defun helper (acc list)
(if list
(helper (1+ acc) (cdr list))
acc))
(defun my-length (list)
(helper 0 list))
The helper function takes two parameters, an accumulation parameter acc, which stores the length of the list so far, and a list list which is the list we're computing the length of.
The important point is that helper is written tail recursively, which means that calling itself is the last thing it does. This means you don't need to allocate a new stack frame every time the function calls itself - since the final result will just be passed all the way back up the chain of stack frames anyway, you can get away with overwriting the old stack frame with a new one, so your recursive function can operate in constant space.
This form of program transformation - from a recursive (but non-tail-recursive) definition to an equivalent definition using a tail-recursive helper function is an important idiom in functional programming - one that it's worth spending a bit of time understanding.
Creating recursive functions to operate on recursive datastructures is indeed good for in lisp. And a list (in lisp) is defined as a recursive datastructure, so you should be ok.
However, as you have experienced, if traversing a datastructure a million items deep using recursion, will also allocate a million frames on the stack, and you can expect a stack overflow unless you specifically ask your runtime environment to allocate huge amount of stack-space (I have no idea if or how you could do this in gnu clisp...).
First of all, I think this shows that the list-datastructure isn't really a the best for everything, and in this case another structure might be better (however, you might not have come to vectors in your lisp-book yet ;-)
Another thing is that for large recursions such as this to be effective, the compiler should optimise tail recursions and convert them into iterations. I don't know if clisp has this functionality, but you would need to change your function to actually be optimisable. (If "tail recursion optimisation" makes no sense, please let me know and I can dig up some references)
For other ways of iterating, take a look at:
http://www.lispworks.com/documentation/HyperSpec/Body/c_iterat.htm
http://www.lispworks.com/documentation/HyperSpec/Body/06_a.htm
Or other datastructures:
http://www.lispworks.com/documentation/HyperSpec/Body/t_vector.htm
(DEFUN nrelem(l)
(if (null l)
0
(+ (nrelem (rest l)) 1)
))