I read about lexical-let's memory leak, for example here:
Are there any problems with lexical-let or other cl-macros??? - Users list for the GNU Emacs text editor - ArchiveOrange
It says:
"Note that variables bound with lexical-let are never released, even
if they are never used. Try
(loop for i from 1 to 100000 collect (lexical-let ((x i)) '()))
and watch it eat memory."
But I think this code eats memory just because the list made by loop grows.
So, I wrote a few elisp codes to check when it occurs but I could not find a example of the leak.
This is how memory grows with time when I execute the code below.
(require 'cl)
(defvar num-loop-1 30)
(defvar num-loop-2 100000)
(loop for i from 1 to num-loop-1 do
(loop for j from 1 to num-loop-2 collect
(lexical-let ((x `(,i ,j))) (lambda () x))))
It looks like there is no leak.
See more examples here:
https://gist.github.com/1703325
ADDED: This is how the first example eats memory. As I said, I think it is an artifact.
I just found this in emacs-devel:
When does Emacs Lisp's lexical-let leak memory?
So... Is it true, that "variables bound with lexical-let are never
released, even if they are never used"?
Not that I know. Of course, this code is not bug-free, but I don't know
of any concrete case that bumps into such a bug.
-- Re: lexical-let cause memory leaks?
Related
In my .emacs file, I have:
(defadvice narrow-to-region (around test activate)
(message "advice")
ad-do-it)
When I call narrow-to-region, the advice runs and prints 'advice' before narrowing.
When I call narrow-to-defun, it does not.
I found where narrow-to-defun is defined - in lisp.el, and re-evaluated the function. At this point, the advice started running.
What could cause this?
The problem is, apparently, due to byte-compilation and therefore the inability to advise the narrowing primitives (narrow-to-region is the primitive, narrow-to-defun calls narrow-to-region).
The following post on Null Program ("The Limits of Emacs Advice") goes into detail about this problem. Here's the shortish version from deep in the post:
It turns out narrow-to-region is so special -- probably because it's used very frequently -- that it gets its own bytecode. The primitive function call is being compiled away into a single instruction. This means my advice will not be considered in byte-compiled code. Darnit. The same is true for widen (code 126).
As to why the advice started working after you re-evaluated narrow-to-defun: I'm guessing it's because you ended up replacing the byte-compiled version when you re-evaluated.
#Dan described the problem well. Here is some info that might help you work around it.
What you can do is to advise (or to redefine) also narrow-to-defun (and perhaps narrow-to-page), so it acts similarly.
FWIW, I do something similar in library wide-n.el (see Multiple Narrowings).
I advise narrow-to-region. But I also redefine narrow-to-defun and narrow-to-page. In all 3 cases I make the same change, to record the details of each narrowing so you can return to them later. Here is the advice, for example:
(defadvice narrow-to-region (before push-wide-n-restrictions activate)
"Push the region limits to `wide-n-restrictions'.
You can use `C-x n x...' to widen to previous buffer restrictions."
(when (or (interactive-p) wide-n-push-anyway-p)
(wide-n-push (ad-get-arg 0) (ad-get-arg 1)))) ; Args START and END.
And here is the relevant part of the narrow-to-defun redefinition:
...
(goto-char end)
(re-search-backward "^\n" (- (point) 1) t)
(when (or (interactive-p) wide-n-push-anyway-p) (wide-n-push beg end)) ; <=====
(narrow-to-region beg end))))
How can I rename elisp macro? To be more accurate, I want make defun to be synonym to cl-defun.
I do not care about time or memory overhead.
Summary
I don't think you can do that - at least not easily.
Since cl-defun expands to defun, you will get an infinite macroexpand loop when using defun if you do the obvious (defalias 'defun 'cl-defun).
The is a way...
So what you need to do is
save the original defun: (fset 'defun-original (symbol-function 'defun)).
copy the definition of cl-defun in cl-macs.el, replacing defun with defun-original.
replace defun with cl-defun using defalias: (defalias 'defun 'cl-defun).
Now, at least, if things go sour, you can restore the original behavior with (fset 'defun (symbol-function 'defun-original)).
...but you don't want it
However, I think you don't really want to do that.
If you want to use a Common Lisp, use it. Trying to pretend that you can turn Elisp into CL will cause you nothing but grief. I tried to travel that road 15 years ago - there is no fun there. It should be easier now, at least there is lexical binding, but I still don't think it is worth the effort.
If you want to extend Emacs, then using cl-defun makes even less sense: your extensions will be useless for others and you won't even be able to ask for help because few people will bother with such a radical change in such a basic functionality for such a tiny gain.
In general, you can make a synonym very simply with (defalias 'foo 'cl-defun).
But the expansion of a call to cl-defun uses defun, so if you do (defalias 'defun 'cl-defun) you'll get into infinite loops.
You can probably get what you want by replacing defun with a macro which either does what defun does or what cl-defun does, depending on whether the cal uses cl-defun features or not. E.g. using advice-add (which is in Emacs's trunk; you can use defadvice in older Emacsen to get simlar results) it could look something like the untested code below:
(defun my-defun-dispatch (doit name args &rest body)
(let ((needs-cl nil))
(dolist (arg args)
(unless (and (symbolp arg)
(or (not (eq ?& (aref (symbol-name arg) 0)))
(memq arg '(&optional &rest))))
(setq needs-cl t)))
(if needs-cl
`(cl-defun ,name ,args ,#body)
(funcall doit name args body))))
(advice-add :around 'defun 'my-defun-dispatch)
Any particular reason?
Maybe it's safe to do, and I'm sure it's possible, but I'm very dubious that you should be attempting it in the first place if you can't figure out how to go about it. I don't mean that as any kind of insult; I just suspect that a fundamental change like this could easily cause problems, so you ought to have a solid handle on elisp first before trying it.
I realise that's a bit of a non-answer, but I thought it was worth saying.
FYI cl-defun is defined in terms of defun.
I am learning Lisp from the book "The Land of Lisp" by Conrad Barski. Now I have hit my first stumbling block, where the author says:
Calling yourself in this way is not only allowed in Lisp, but is often
strongly encouraged
after showing the following example function to count the items in a list:
(defun my-length (list)
(if list
(1+ (my-length (cdr list)))
0))
When I call this function my-length with a list containing a million items, I get a stack overflow error. So either you never expect to have a list that long in Lisp (so maybe my use case is useless) or there is another way to count items in such a long list. Can you maybe shine some light on this? (I am using GNU CLISP on Windows, by the way).
TCO (tail call optimization) in CLISP using the example from Chris Taylor:
[1]> (defun helper (acc list)
(if list
(helper (1+ acc) (cdr list))
acc))
(defun my-length (list)
(helper 0 list))
HELPER
Now compile it:
[2]> (compile 'helper)
MY-LENGTH
[3]> (my-length (loop repeat 100000 collect t))
*** - Program stack overflow. RESET
Now, above does not work. Let's set the debug level low. This allows the compiler to do TCO.
[4]> (proclaim '(optimize (debug 1)))
NIL
Compile again.
[5]> (compile 'helper)
HELPER ;
NIL ;
NIL
[6]> (my-length (loop repeat 100000 collect t))
100000
[7]>
Works.
Allowing the Common Lisp compiler to do TCO is most often controlled by the debug level. With a high debug level, the compiler generates code which uses a stack frame for each function call. This way each call can be traced and will be seen in a backtrace. With a lower debug level the compiler may replace tail calls with jumps in the compiled code. These calls then will not be seen in a backtrace - which usually makes debugging harder.
You're looking for tail recursion. At the moment your function is defined like
(defun my-length (list)
(if list
(1+ (my-length (cdr list)))
0))
Notice that after my-length has called itself, it needs to add one to the result before passing that value to its calling function. This need to modify the value before returning it means that you need to allocate a new stack frame for every call, the the space used is proportional to the length of the list. This is what causes a stack overflow on long lists.
You can re-write it to use a helper function
(defun helper (acc list)
(if list
(helper (1+ acc) (cdr list))
acc))
(defun my-length (list)
(helper 0 list))
The helper function takes two parameters, an accumulation parameter acc, which stores the length of the list so far, and a list list which is the list we're computing the length of.
The important point is that helper is written tail recursively, which means that calling itself is the last thing it does. This means you don't need to allocate a new stack frame every time the function calls itself - since the final result will just be passed all the way back up the chain of stack frames anyway, you can get away with overwriting the old stack frame with a new one, so your recursive function can operate in constant space.
This form of program transformation - from a recursive (but non-tail-recursive) definition to an equivalent definition using a tail-recursive helper function is an important idiom in functional programming - one that it's worth spending a bit of time understanding.
Creating recursive functions to operate on recursive datastructures is indeed good for in lisp. And a list (in lisp) is defined as a recursive datastructure, so you should be ok.
However, as you have experienced, if traversing a datastructure a million items deep using recursion, will also allocate a million frames on the stack, and you can expect a stack overflow unless you specifically ask your runtime environment to allocate huge amount of stack-space (I have no idea if or how you could do this in gnu clisp...).
First of all, I think this shows that the list-datastructure isn't really a the best for everything, and in this case another structure might be better (however, you might not have come to vectors in your lisp-book yet ;-)
Another thing is that for large recursions such as this to be effective, the compiler should optimise tail recursions and convert them into iterations. I don't know if clisp has this functionality, but you would need to change your function to actually be optimisable. (If "tail recursion optimisation" makes no sense, please let me know and I can dig up some references)
For other ways of iterating, take a look at:
http://www.lispworks.com/documentation/HyperSpec/Body/c_iterat.htm
http://www.lispworks.com/documentation/HyperSpec/Body/06_a.htm
Or other datastructures:
http://www.lispworks.com/documentation/HyperSpec/Body/t_vector.htm
(DEFUN nrelem(l)
(if (null l)
0
(+ (nrelem (rest l)) 1)
))
The reason I don't understand why Emacs 24's new lexical scoping features are that great is that I can't think of any new functionality that couldn't have been implemented without them. For example, the following closure:
(setq lexical-binding t)
(defun f1 (num1)
(lambda (num2)
(setq num1 (* num1 num2))))
(fset 'f2 (f1 5))
==> (closure ((num1 . 5) t) (num2) (setq num1 (* num1 num2)))
(f2 5)
==> 25
(f2 2)
==> 50
Can be implemented with regular dynamic scoping like so:
(defun f1 (num)
(let ((tmpvar (make-symbol "num")))
(set tmpvar num)
`(lambda (num2)
(set ',tmpvar (* (eval ,tmpvar) num2)))))
(fset 'f2 (f1 5))
==> (lambda (num2) (set (quote num) (+ (eval num) num2)))
(f2 5)
==> 25
(f2 2)
==> 50
(fset 'f3 (f1 9))
==> (lambda (num2) (set (quote num) (+ (eval num) num2)))
(f3 3)
==> 27
(f3 2)
==> 54
(f2 10)
==> 500
Okay, so not all languages have something analogous to elisp's uninterned symbols, so I understand why lexical scoping is so great in their case. But what about elisp? Can you think of anything that I can do now (as of Emacs 24) that I couldn't do before, thanks to lexical scoping?
You don't need uninterned symbols, use cons instead of make-symbol, car instead of eval, and setcar instead of set and your example will work just as well (and be more efficient).
Note also that the progression from machine-language to ever higher-level languages has been mostly based on making more and more things impossible (or at least much harder). Of course those facilities taken away from the programmers were rarely used and/or considered too dangerous. Think of using uninitialized variables (possible in C, but impossible in Java and many other languages), or jumping into the middle of an instruction.
As for some downside of your example code: not only it's less readable, but the compiler basically won't be able to know that you're constructing code, so it won't be allowed to look inside that "`(lambda ...)" to compile it, expand its macro calls, give you warnings about suspicious elements, ...
There have always been workarounds to simulate lexical binding in Emacs, so it's not so much about being able to do new things.
The manual says:
Lexical binding opens up a lot more opportunities for optimization, so
Emacs Lisp code that makes use of lexical binding is likely to run
faster in future Emacs versions. Such code is also much more friendly
to concurrency, which we want to add to Emacs in the near future.
I think that is the primary benefit.
The flip side to this is that dynamic binding was purposely chosen when Emacs was written for good reasons which still hold true today, and so lexical binding should certainly not be considered the New Way of doing things.
Global variables are generally considered a bad idea in programming, but Emacs is an unusual case where this doesn't really apply, because much of its immense flexibility -- one of the key things that makes Emacs great -- derives directly from dynamic binding. Without dynamic binding, it would not be possible to bend the application to the requirements of the individual user to anything like the extent that Emacs allows.
In my opinion, lexical binding should be used cautiously and only for variables for which another user could not conceivably find a reason to override. By default variables should be defvar'd so that the ability to customize the behaviour (even in ways that the author did not anticipate) is preserved.
I think implementations of OO programming tend to fit well with lexical scope. An object with state maps rather directly to a lexical closure.
I'm sure that the CLOS implementation in common lisp leverages lexical scope heavily. It's hard for me to imagine how that spec could be implemented with dynamic scope only, but I'm sure it's possible.
I've done a bit of research on this subject and am turning up blanks. There seem to be implementation-dependent ways of doing Unix signal handling in Common Lisp, but is there a package that gives a cross-implementation way of doing signal handling?
I would mainly like to listen for SIGINT and do a graceful shutdown in my app. I'm using Clozure CL 1.7 on linux...like mentioned, it would be great for a package for this, but if I have to resort to implementation-specific code, that's fine.
I'm also not completely married to using SIGINT (although it's ideal). I can use another signal if needed.
If this is going to be messy, does anyone have any other suggestions for gracefully shutting down a lisp app from outside the app? One idea I had is to create a file the app monitors for, and if it detects the file, it shuts down...kind of hacky, though.
Thanks!
Although out of ignorance I was originally skeptical of Daimrod's comment (first comment under the question) about using CFFI, I looked around a bit more and found http://clozure.com/pipermail/openmcl-devel/2010-July/011675.html. I adapted it to use CFFI and have confirmed this works on SBCL/CCL/clisp (probably others) on linux pretty damn well:
(defmacro set-signal-handler (signo &body body)
(let ((handler (gensym "HANDLER")))
`(progn
(cffi:defcallback ,handler :void ((signo :int))
(declare (ignore signo))
,#body)
(cffi:foreign-funcall "signal" :int ,signo :pointer (cffi:callback ,handler)))))
(set-signal-handler 2
(format t "Quitting lol!!!11~%")
;; fictional function that lets the app know to quit cleanly (don't quit from callback)
(signal-app-to-quit))
Note that from what I understand, whatever is in the body of the callback must be short and sweet! No lengthy processing. In the linked article, the macro actually creates a separate thread just for handling the signal, which is overkill for my purposes, since I'm just setting a global variable from nil to t and returning.
Anyway, hopefully this is helpful to others!
I can't find a general library for signal handling either. However, Slime implements "create a custom SIGINT handler" for most Lisp implementations. By looking at the CCL case of that code, I found ccl:*break-hook*. ccl:*break-hook* is not in the documentation, but the commit it was introduced in is located here.
This trivial example code works on my system (CCL 1.8, linux x86):
(setf ccl:*break-hook*
(lambda (cond hook)
(declare (ignore cond hook))
(format t "Cleaning up ...")
(ccl:quit)))
After this code is entered into a non-Slime REPL, sending SIGINT will cause the program to print "Cleaning up ..." and exit.
This is a late answer, but for anybody else searching for this, have a look at trivial-signal, available on Quicklisp. This is based on CFFI.
Example
(signal-handler-bind ((:int (lambda (signo)
(declare (ignorable signo))
...handler...)))
...body...)
If you use SBCL, you cannot change the signal mask without causing SBCL to crash. Ask nyef about his tips on how to fix SBCL...
So there is trivial-signal as mentioned. Here's how I catch a C-c in my code:
(handler-case
(my-app-main-function)
;; AFAIK trivial-signal is supposed to handle the implementation differences.
(#+sbcl sb-sys:interactive-interrupt
#+ccl ccl:interrupt-signal-condition
#+clisp system::simple-interrupt-condition
#+ecl ext:interactive-interrupt
#+allegro excl:interrupt-signal
() (progn
(format *error-output* "Aborting.~&")
(exit)))
(error (c) (format t "Woops, an unknown error occured:~&~a~&" c)))