This question already has answers here:
What makes Lisp macros so special?
(15 answers)
Closed 3 months ago.
I keep reading that Lisp macros are one of the most powerful features of the language. But reading over the specifications and manuals, they are just functions whose arguments are unevaluated.
Given any macro (defmacro example (arg1 ... argN) (body-forms)) I could just write (defun example (arg1 ... argN) ... (body-forms)) with the last body-form turned into a list and then call it like (eval (example 'arg1 ... 'argN)) to emulate the same behavior of the macro. If this were the case, then macros would just be syntactic sugar, but I doubt that syntactic sugar would be called a powerful language feature. What am I missing? Are there cases where I cannot carry out this procedure to emulate a macro?
I can't talk about powerful because it can be a little bit subjective, but macros are regular Lisp functions that work on Lisp data, so they are as expressive as other functions. This isn't the case with templates or generic functions in other languages that rely more on static types and are more restricted (on purpose).
In some way, yes macros are simple syntactic facilities, but you are focused in your emulation on the dynamic semantics of macros, ie. how you can run code that evaluates macros at runtime. However:
the code using eval is not equivalent to expanded code
the preprocessing/compile-time aspect of macros is not emulated
Lexical scope
Function, like +, do not inherit the lexical scope:
(let ((x 30))
(+ 3 4))
Inside the definition of +, you cannot access x. Being able to do so is what "dynamic scope" is about (more precisely, see dynamic extent, indefinite scope variables). But nowadays it is quite the exception to rely on dynamic scope. Most functions use lexical scope, and this is the case for eval too.
The eval function evaluates a form in the null lexical environment, and it never has access to the surrounding lexical bindings. As such, it behaves like any regular function.
So, in you example, calling eval on the transformed source code will not work, since arg1 to argnN will probably be unbound (it depends on what your macro does).
In order to have an equivalent form, you have to inject bindings in the transformed code, or expand at a higher level:
(defun expand-square (var)
(list '* var var))
;; instead of:
(defun foo (x) (eval (expand-square 'x))) ;; x unbound during eval
;; inject bindings
(defun foo (x) (eval `(let ((z ,x)) (expand-square z))))
;; or expand the top-level form
(eval `(defun foo (x) ,(expand-square 'x)))
Note that macros (in Common Lisp) also have access to the lexical environment through &environment parameters in their lambda-list. The use of this environment is implementation dependent, but can be used to access the declarations associated with a variable, for example.
Notice also how in the last example you evaluate the code when defining the function, and not when running it. This is the second thing about macro.
Expansion time
In order to emulate macros you could locally replace a call to a macro by a form that emulates it at runtime (using let to captures all the bindings you want to see inside the expanded code, which is tedious), but then you would miss the useful aspect of macros that is: generating code ahead of time.
The last example above shows how you can quote defun and wrap it in eval, and basically you would need to do that for all functions if you wanted to emulate the preprocessing work done by macros.
The macro system is a way to integrate this preprocessing step in the language in a way that is simple to use.
Conclusion
Macros themselves are a nice way to abstract things when functions can't. For example you can have a more human-friendly, stable syntax that hides implementation details. That's how you define pattern-matching abilities in Common Lisp that make it look like they are part of the language, without too much runtime penalty or verbosity.
They rely on simple term-rewriting functions that are integrated in the language, but you can emulate their behavior either at compile-time or runtime yourself if you want. They can be used to perform different kinds of abstraction that are usually missing or more cumbersome to do in other languages, but are also limited: they don't "understand" code by themselves, they don't give access to all the facilities of the compiler (type propagation, etc.). If you want more you can use more advanced libraries or compiler tools (see deftransform), but macros at least are portable.
Macros are not just functions whose arguments are unevaluated. Macros are functions between programming languages. In other words a macro is a function whose argument is a fragment of source code of a programming language which includes the macro, and whose value is a fragment of source code of a language which does not include the macro (or which includes it in a simpler way).
In very ancient, very rudimentary, Lisps, before people really understood what macros were, you could simulate macros with things called FEXPRs combined with EVAL. A FEXPR was simply a function which did not evaluate its arguments. This worked in such Lisps only because they were completely dynamically scoped, and the cost of it working was that compilation of such things was not possible at all. Those are two enormous costs.
In any modern Lisp, this won't work at all. You can write a toy version of FEXPRs as a macro (this may be buggy):
(defmacro deffex (fx args &body body)
(assert (every (lambda (arg)
(and (symbolp arg)
(not (member arg lambda-list-keywords))))
args)
(args) "not a simple lambda list")
`(defmacro ,fx ,args
`(let ,(mapcar (lambda (argname argval)
`(,argname ',argval))
',args (list ,#args))
,#',body)))
So now we could try to write a trivial binding construct I'll call with using this thing:
(deffex with (var val form)
(eval `(let ((,var ,val)) ,form)))
And this seems to work:
> (with a 1 a)
1
Of course, we're paying the cost that no code which uses this construct can ever be compiled so all our programs will be extremely slow, but perhaps that is a cost we're willing to accept (it's not, but never mind).
Except, of course, it doesn't work, at all:
> (with a 1
(with b 2
(+ a b)))
Error: The variable a is unbound.
Oh dear.
Why doesn't it work? It doesn't work because Common Lisp is lexically scoped, and eval is a function: it can't see the lexical bindings.
So not only does this kind of approach prevent compilation in a modern Lisp, it doesn't work at all.
People often, at this point, suggest some kind of kludge solution which would allow eval to be able to see lexical bindings. The cost of such a solution is that all the lexical bindings need to exist in compiled code: no variable can ever be compiled away, not even its name. That's essentially saying that no good compilers can ever be used, even for the small part of your programs you can compile at all in a language which makes extensive use of macros like CL. For instance, if you ever use defun you're not going to be able to compile the code in its body. People do use defun occasionally, I think.
So this approach simply won't work: it worked by happenstance in very old Lisps but it can't work, even at the huge cost of preventing compilation, in any modern Lisp.
More to the point this approach obfuscates the understanding of what macros are: as I said at the start, macros are functions between programming languages, and understanding that is critical. When you are designing macros you are implementing a new programming language.
I'm starting to work through SICP using DrRacket. I installed the sicp package, and declared #lang sicp at the top of the unit test file and then (require rackunit "xxx.scm"), but I get an unbound identifier error. Is there a mistake, or is it not possible to use rackunit with the sicp package in this way?
You need to use #%require.
#%require is actually a primitive type in the lowest level of racket and it is slightly different than require:
#lang sicp
(#%require rackunit "xxx.scm")
The file you want to test becomes a module so you can use it from other code by providing the identifiers you want to expose:
(#%provide procedure-name)
You can also just require a few needed forms. eg. error and time from racket/base:
(#%require (only racket/base error time))
A hint on finding out where they are is to search the online manuals or from Help > Racket documentation in DrRacket. Eg. here is an example of searching error where you have many choices, but the racket prefixed ones are the ones you're looking for.
NB: Not all forms are compatible across languages. Eg. R5RS has different pair implementation than #lang racket
I would like to create a new language that has the same syntax as typed racket, but when executed will do two things:
run the given program as typed racket
if type checks, then translates the input into another language (say python). I am planning to write a translator from typed racket forms to python syntax.
Any suggestions on how to start or pointers to some skeleton code? I have read this tutorial but it mostly talks about creating new syntax which I don't need in my case.
I understand how I can write racket code to translate from one language to another, what I don't get is how I can do both of the above, i.e., first run it as another language, and then do something else with the same input.
It sounds like you want to make a language with a new #%module-begin form. This form is either inserted by the reader (when you do a #lang .... line at the top of your file, or by the language if you wrote the module out by hand. Either way, its generally associated with your language definition. This macro has full access to the entire module's unexpanded syntax. So for example, a macro like this:
(provide (rename-out [-module-begin #%module-begin]))
(define-simple-macro (-module-begin body ...)
(#%module-begin
(writeln '(body ...))
body ...))
Would create a language that does two things:
Print out the body of the code (as an s-expression), and
Run the body with the #%module-begin that the language definition was written in.
You can see how you can use this technique to grab the body of your program twice, and do two different things with it. So let's try running this example. First, lets take the sample from above, and put it in a file "mylang.rkt":
#lang racket
(provide (rename-out [-module-begin #%module-begin])
(except-out (all-from-out racket) #%module-begin))
(require syntax/parse/define)
(define-simple-macro (-module-begin body ...)
(#%module-begin
(writeln '(body ...))
body ...))
And now we can write a program in mylang like this:
#lang s-exp "mylang.rkt"
(+ 1 2)
And when you run it, you'll get something like this:
((+ 1 2))
3
First its printing out the program text, then running it.
You can read more about this process in a paper I wrote discussing this aspect of the Video language. You might also find the book Beautiful Racket to have some examples you might find useful.
Lately I'm learning Racket, and I'm having some difficulties with understanding the "define-type" syntax. I've tried the following code:
#lang racket
(define-type Num Number)
but it outputs the following error message:
define-type: unbound identifier in module in: define-type
May anyone help me dealing this error? I have read all possible documentation and it seems like it should be working.
There is a reason every Racket program starts with a line beginning with #lang: Racket is not just a programing language, but an ecosystem of programming languages. Every file (and more specifically, every module) can be in its own programming language, and they can all talk to each other with ease.
By default, Dr. Racket creates new files with a #lang racket line at the top. This is the “Racket language”, but it is not the only language provided by the “Racket system”, which actually includes dozens of languages, some not too different from #lang racket, others almost entirely unrelated.
When you want to use Typed Racket, you need to opt in to using that language instead of ordinary #lang racket, which is dynamically typed. You can do this by writing #lang typed/racket at the top of your program.
#lang typed/racket
(define-type Num Number)
Now all the features of Typed Racket will be available to you within that module.
Scheme macros, at least the syntax-case variety, are said to allow arbitrary computation on the code to be transformed. However (both in the general case and in the specific case I'm currently looking at) this requires the computation to be specified in terms of recursive functions. When I try various variants of this, I get e.g.
main.scm:32:71: compile: unbound identifier in module (in the transformer environment, which does not include the run-time definition) in: expand-vars
(The implementation is Racket, if it matters.)
The upshot seems to be that you can't define named functions until after macro processing.
I suppose I could resort to the Y combinator, but I figure it's worth asking first whether there's a better approach?
Yes, the fact that you're using Racket matters -- in Racket, there is something that is called "phase separation", which means that the syntax level cannot use runtime functions. For example, this:
#lang racket
(define (bleh) #'123)
(define-syntax (foo stx)
(bleh))
(foo)
will not work since bleh is bound at a runtime, not available for syntax. Instead, it should be
(define-for-syntax (bleh) #'123)
or
(begin-for-syntax (define (bleh) #'123))
or moved as an internal definition to the macro body, or moved to its own module and required using (require (for-syntax "bleh.rkt")).