The trace macro is very useful for debugging. But it comes to a halt, when used upon any macro. Like if I try to do the following :
CL-USER> (trace push)
Then, it'll give an error saying:
can't use encapsulation to trace anonymous function #<FUNCTION (MACRO-FUNCTION
PUSH) {100053FB9B}>
[Condition of type SIMPLE-ERROR]
Well, that's obvious because the clhs page of trace, clearly defines it upon functions. So, what is the reason for not having any facility for tracing macros in Common Lisp?
Is there any other (unconventional) way to trace macros in Common Lisp?
The Common Lisp standard only mentions tracing of functions. In compiled implementations, macro expansion usually takes place at compile time and thus tracing of macros is usually not supported.
But some Common Lisp implementations can trace macros via a Lisp interpreter (!):
CLISP can trace macros:
[1]> (defmacro foo (a) a)
FOO
[2]> (trace foo)
;; Tracing macro FOO.
(FOO)
[3]> (loop for i below 4 collect (foo i))
1. Trace: (FOO I)
1. Trace: FOO ==> I
1. Trace: (FOO I)
1. Trace: FOO ==> I
1. Trace: (FOO I)
1. Trace: FOO ==> I
1. Trace: (FOO I)
1. Trace: FOO ==> I
(0 1 2 3)
LispWorks is another implementation which supports tracing of macros.
So, what is the reason for not having any facility for tracing macros in Common Lisp?
As mentioned that's the language standard. Beyond the language standard implementations provide all kinds of language extensions in various ways, including the ability of some Lisp interpreters (!) to trace macros.
If the code is already compiled, tracing won't work anyway. Having a Lisp interpreter helps, but implementations are not required to have an interpreter. Lisp interpreter here means an executing engine which works from Lisp code as data.
Using trace on a macro seems a little odd, but it works in CLISP:
(trace push)
(defparameter *stack* '())
(defun push-xy (x y)
(push x *stack*)
(push y *stack*))
; 1. Trace: (push x *stack*)
; 1. Trace: push ==> (setq *stack* (cons x *stack*))
; 1. Trace: (push y *stack*)
; 1. Trace: push ==> (setq *stack* (cons y *stack*))
; ==> push-xy
The standard does not say when it should expand macros so this might happen when functions and lambdas are defined, compiled and sometimes called. Some implementations run the macro twice so you get double the output.
I never use this. I rather use macroexpand-1:
(macroexpand-1 '(push x *stack)))
; ==> (setq *stack (cons x *stack))
; ==> t
If your form returns a new form that uses macros you might want to try macroexpand instead. Its like calling macroexpand-1 over and over until there are no transformation left.
*macroexpand-hook* is the expected method for tracing macros.
http://www.lispworks.com/documentation/HyperSpec/Body/v_mexp_h.htm#STmacroexpand-hookST
Related
I may be asking for the impossible, but am wondering nonetheless.
Is it possible to obtain an analog of the stack-trace for macros? That is, if one set a break-point inside a certain function, the macro-stack-trace would list all macros (perhaps with their inputs) that were macroexpanded to get to that level in the code.
From what I understand, this is currently impossible, but it may be due to my shallow understanding. Does Allegro or SBCL allow or track this kind of information? It appears that this would be really useful for debugging macros.
Any help or advice is appreciated.
As SBCL is a compiler-only implementation meaning all code is automatically compiled (in contrast to being "interpreted"). Calls to macros are expanded as part of compilation, so the fact that something was a macro call is lost.
(defmacro m (n)
`(/ 10 ,n))
(defun foo (x) (m x))
SBCL:
* (foo 0)
debugger invoked on a DIVISION-BY-ZERO in thread
#<THREAD "main thread" RUNNING {1001E06493}>:
arithmetic error DIVISION-BY-ZERO signalled
Operation was /, operands (10 0).
Type HELP for debugger help, or (SB-EXT:EXIT) to exit from SBCL.
restarts (invokable by number or by possibly-abbreviated name):
0: [ABORT] Exit debugger, returning to top level.
(SB-KERNEL::INTEGER-/-INTEGER 10 0)
0] backtrace
Backtrace for: #<SB-THREAD:THREAD "main thread" RUNNING {1001E06493}>
0: (SB-KERNEL::INTEGER-/-INTEGER 10 0)
1: (FOO 0)
2: (SB-INT:SIMPLE-EVAL-IN-LEXENV (FOO 0) #<NULL-LEXENV>)
3: (EVAL (FOO 0))
4: (INTERACTIVE-EVAL (FOO 0) :EVAL NIL)
[...]
Some implementations, e.g. Allegro CL, support both interpreted as well as compiled code, the first being helpful in debugging, the second giving better performance. (I show here the command-line interactions. Allegro also offers a GUI to set breakpoints that I'm not familiar with.)
cl-user(4): (foo 0)
Error: Attempt to divide 10 by zero.
[condition type: division-by-zero]
Restart actions (select using :continue):
0: Return to Top Level (an "abort" restart).
1: Abort entirely from this (lisp) process.
[1] cl-user(5): :zoom
Evaluation stack:
(error division-by-zero :operation ...)
->(/ 10 0)
(foo 0)
(eval (foo 0))
[...]
The zoom command takes many options to be more verbose, this shows the form (block foo (m x)):
[1] cl-user(6): :zoom :all t
Evaluation stack:
... 4 more newer frames ...
((:runsys "lisp_apply"))
[... sys::funcall-tramp ]
(excl::error-from-code 17 nil ...)
(sys::..runtime-operation "integer_divide" :unknown-args)
(excl::/_2op 10 0)
->(/ 10 0)
[... excl::eval-as-progn ]
(block foo (m x))
(foo 0)
(sys::..runtime-operation "comp_to_interp" 0)
[... excl::%eval ]
(eval (foo 0))
When you (compile 'foo) the macro calls will be expanded away (like for SBCL) and not show up in backtraces anymore (but Allegro's source-level debugging could help).
In general when it comes to defining macros, to help debugging try to expand into function calls and not big bodies of code. E.g. instead of:
(defmacro with-foo ((var-x var-y thing) &body body)
`(let ((,var-x (..derive from ,thing ..))
(,var-y (..derive from ,thing ..)))
,#body))
I would write it like:
(defmacro with-foo ((var-x var-y thing) &body body)
`(call-with-foo (lambda (,var-x ,var-y) ,#body) ,thing))
(defun call-with-foo (func thing)
(let ((x (..derive from thing ..)
(y (..derive from thing ..))
(funcall func x y)))
so it ends up in the stack trace and is easy to redefine.
See this great post by Kent Pitman:
Incidentally, too, back to CL, you should know that when I write these
WITH-xxx macros, I almost always accompany them with a CALL-WITH-xxx
so that I can do either kind of call. But I find I almost never use
the CALL-WITH-xxx even when I was the one to provide it as an option.
The main reason I write them is not to use them but to make
redefinition easier, since I can redefine the CALL-WITH-xxx without
redefining the macro, and so I don't have to recompile the callers if
the definition changes.
Yes, AllegroCl supports tracing and in general debugging of macros. Quite an effort for not sure how much benefit, but Franz tends to do good things to make CL more viable. Pro tip: there is a an option to turn off what I think they call source-level debugging of macros, and you will want to do that if your code makes heavy use of macros or compilation times can get crazy. Just turn it back on when you think you need the source debugging.
I have sample code like this:
#!/usr/bin/guile -s
!#
(define (process body)
(list 'list (map (lambda (lst)
(list 'quote (car lst)))
body)))
(defmacro macro (body)
(list 'quote (process body)))
(display (macro ((foo bar) (bar baz))))
(newline)
it run but I've got error from compiler
ERROR: Unbound variable: process
functions inside macros should be allowed, why I got this error?
Functions inside macros are allowed in Guile and in most other Scheme dialects.
However, the crucial question is: Which functions are available for a macro to call out during the expansion process?
Think of it this way: When the compiler is processing your code, it is first focused on turning your source code into something that can be run at some point in the future. But the compiler might not necessarily be able to execute those same functions right now while it is compiling them, at the same time that your macro is running and expanding the source code in general.
Why wouldn't such a function be available? Well, one example would be: What if the function's body used the macro you are in the midst of defining? Then you would have a little chicken/egg problem. The function would need to run the macro to be compiled (since the macro use in the body needs to be expanded, at compile-time) ... But the macro would need the compiled function available in order to even run!
(Furthermore, there might be some functions that you only want to be available at compile-time, as a helper for your macros, but that you do not want to be available at run-time, so that it will not be included in your program executable when you deploy it, as that would waste space in the deployed binary.)
One of my favorite papers describing this problem, and the particular solution adopted by MzScheme (now known as Racket), is the "You Want It When" paper by Matthew Flatt.
So, this is a problem that any Scheme dialect with a procedural macro system has to deal with in some way, and Guile is no exception.
In Guile's case, one fix that is directly documented in the Guile manual is to use the eval-when special form, which allows you to specify at what phases a particular definition is meant to be available.
(The "You Want It When" paper referenced above describes some problems with eval-when, but since it is what the Guile manual documents, I'm going to stick with it for now. I do recommend that after you understand eval-when, that you then look into Racket's solution, and see if Guile offers anything similar.)
So in your case, since you want the process function to be available at compile-time (for use in the macro definition), you could write:
#!/usr/bin/guile -s
!#
(eval-when (expand)
(define (process body)
(list 'list (map (lambda (lst)
(list 'quote (car lst)))
body))))
(defmacro macro (body)
(list 'quote (process body)))
(display (macro ((foo bar) (bar baz))))
(newline)
I created a function that returns a function in Elisp:
(defun singleton-set (elem)
(defun f (n) (= n elem))
f)
I try to run this in IELM, and it fails:
ELISP> (singleton-set 5)
*** Eval error *** Symbol's value as variable is void: f
ELISP> ((singleton-set 5) 5)
*** Eval error *** Invalid function: (singleton-set 5)
Due to What is the difference between Lisp-1 and Lisp-2? i changed the code to
(defun singleton-set (elem)
(defun f (n) (= n elem))
#'f)
And invocation to (funcall (singleton-set 5) 5), but now the error is
*** Eval error *** Symbol's value as variable is void: elem
I understand from elisp: capturing variable from inner function that this is due to dynamic binding of Emacs Lisp.
How to make functions returning functions possible in Emacs Lisp? What is the reason this mechanism is different from other languages like Python, Scala or Clojure?
Related questions:
elisp functions as parameters and as return value
Elisp interactive function name
How to Create a Temporary Function in Emacs Lisp
In elisp, how do I put a function in a variable?
From the NEWS for Emacs 24:
Lisp changes in Emacs 24.1
Code can now use lexical scoping by default instead of dynamic scoping.
The lexical-binding variable enables lexical scoping for local
variables. It is typically set via a file-local variable in the first
line of the file, in which case it applies to all the code in that
file.
So, in Emacs 24:
(setq lexical-binding t)
(defun singleton-set (elem) (lambda (n) (= n elem)))
(mapcar (singleton-set 1) '(0 1 2 3))
===> (nil t nil nil)
How to make functions returning functions possible in Emacs Lisp?
Using fake closures, and lexical-let.
What is the reason this mechanism is different from other languages like Python, Scala or Clojure?
Richard Stallman answered this question in a paper he wrote a while ago.
(defun singleton-set (elem)
`(lambda (n) (= n ,elem))
See: elisp functions as parameters and as return value
This question already has answers here:
What can you do with Lisp macros that you can't do with first-class functions?
(8 answers)
Closed 5 years ago.
In my quest to fully understand the so powerful lisp macros a question came to my mind. I know that a golden rule about macros is the one saying "Never use a macro when a function will do the work".
However reading Chapter 9 - Practical: Building a Unit Test Framework - from the book Practical Common Lisp I was introduced to the below macro whose purpose was to get rid of the duplication of the test case expression, with its attendant risk of mislabeling of results.
;; Function defintion.
(defun report-result (result form)
(format t "~:[FAIL~;pass~] ... ~a~%" result form))
;; Macro Definition
(defmacro check (form)
`(report-result ,form ',form))
OK, I understand its purpose but I could have done it using a function instead of a macro, for instance:
(setf unevaluated.form '(= 2 (+ 2 3)))
(defun my-func (unevaluated.form)
(report-result (eval unevaluated.form) unevaluated.form))
Is this only possible because the given macro is too simple ?
Furthermore, is Lisp Macro System so powerful relatively its opponents due to the code itself - like control structures, functions, etc - is represented as a LIST ?
But if it were a macro you, could have done:
(check (= 2 (+ 2 3)))
With a function, you have to do:
(check '(= 2 (+ 2 3)))
Also, with the macro the (= 2 (+ 2 3)) is actually compiled by the compiler, whereas with the function it's evaluated by the eval function, not necessarily the same thing.
Addenda:
Yes, it's just evaluating the function. Now what that means is dependent upon the implementation. Some can interpret it, others can compile and execute it. But the simple matter is that you don't know from system to system.
The null lexical environment that others are mentioning is also a big deal.
Consider:
(defun add3f (form)
(eval `(+ 3 ,form)))
(demacro add3m (form)
`(+ 3 ,form))
Then observe:
[28]> (add3m (+ 2 3))
8
[29]> (add3f '(+ 2 3))
8
[30]> (let ((x 2)) (add3m (+ x 3)))
8
[31]> (let ((x 2)) (add3f '(+ x 3)))
*** - EVAL: variable X has no value
The following restarts are available:
USE-VALUE :R1 Input a value to be used instead of X.
STORE-VALUE :R2 Input a new value for X.
ABORT :R3 Abort main loop
Break 1 [32]> :a
That's really quite damning for most use cases. Since the eval has no lexical environment, it can not "see" the x from the enclosing let.
The better substitution would be not with eval, which won't perform as expected for all cases (for example, it doesn't have access to the lexical environment), and is also overkill (see here: https://stackoverflow.com/a/2571549/977052), but something using anonymous functions, like this:
(defun check (fn)
(report-result (funcall fn) (function-body fn)))
CL-USER> (check (lambda () (= 2 (+ 2 3))))
By the way, this is how such things are accomplished in Ruby (anonymous functions are called procs there).
But, as you see, it becomes somewhat less elegant (unless you add syntax sugar) and, there's actually a bigger problem: ther's no function-body function in Lisp (although there may be non-standard ways to get at it). Overall, as you see, for this particular task the alternative solutions are substantially worse, although in some cases such approach could work.
In general, though, if you want to do something with the source code of the expressions passed into the macro (and usually this is the primary reason of using macros), functions would not be sufficient.
The report-result function needs both the source code and the result of the execution.
The macro CHECK provides both from a single source form.
If you put a bunch of check forms into the file, they are easily compiled using the usual process of compiling Lisp files. You'll get a compiled version of the checking code.
Using a function and EVAL (better use COMPILE) you would have deferred the source evaluation to a later time. It would also not be clear if it is interpreted or compiled. In case of compilation, you would then later get the compiler's checks.
As suggested in a macro-related question I recently posted to SO, I coded a macro called "fast" via a call to a function (here is the standalone code in pastebin):
(defun main ()
(progn
(format t "~A~%" (+ 1 2 (* 3 4) (+ 5 (- 8 6))))
(format t "~A~%" (fast (+ 1 2 (* 3 4) (+ 5 (- 8 6)))))))
This works in the REPL, under both SBCL and CMUCL:
$ sbcl
This is SBCL 1.0.52, an implementation of ANSI Common Lisp.
...
* (load "bug.cl")
22
22
$
Unfortunately, however, the code no longer compiles:
$ sbcl
This is SBCL 1.0.52, an implementation of ANSI Common Lisp.
...
* (compile-file "bug.cl")
...
; during macroexpansion of (FAST (+ 1 2 ...)). Use *BREAK-ON-SIGNALS* to
; intercept:
;
; The function COMMON-LISP-USER::CLONE is undefined.
So it seems that by having my macro "fast" call functions ("clone","operation-p") at compile-time, I trigger issues in Lisp compilers (verified in both CMUCL and SBCL).
Any ideas on what I am doing wrong and/or how to fix this?
Some remarks about your code.
multiple tests of an object for equality can be replaced by MEMBER
backquote with a following comma does nothing. You can just remove that.
you can ensure that your functions are available for a macro by a) moving these functions to an additional file and compile/load that before use of the macro, by b) using EVAL-WHENto inform the compiler to evaluate the definition of the functions or by c) adding the functions to the macro as local functions
Example:
(defmacro fast (&rest sexpr)
(labels ((operation-p (x)
(member x '(+ - * /)))
(clone (sexpr)
(if (consp sexpr)
(destructuring-bind (head . tail) sexpr
(if (operation-p head)
`(the fixnum (,head ,#(clone tail)))
(cons (clone head) (clone tail))))
sexpr)))
(car (clone sexpr))))
Note that this and your version of FAST are not complete code walkers. They recognize only simple function calls (and not the other Lisp constructs like LAMBDA, LET, FLET, LABELS, etc.).
Never mind, I figured it out: I had to move the functions invoked by the macro (and therefore required during compilation) in a separate file, "compile-file" it first, "load" it, then "compile-file" the one with the macro.
Macro-expansion tie happens (typically) during compile time.
That means that any functions used during the macro expansion (note, not necessarily in the macro-expansion, the return value as it were) must be defined when the macro is encountered during compilation.