I've been working through Practical Common Lisp and as an exercise decided to write a macro to determine if a number is a multiple of another number:
(defmacro multp (value factor)
`(= (rem ,value ,factor) 0))
so that :
(multp 40 10)
evaluates to true whilst
(multp 40 13)
does not
The question is does this macro leak in some way? Also is this "good" Lisp? Is there already an existing function/macro that I could have used?
Siebel gives an extensive rundown (for simple cases anyway) of possible sources of leaks, and there aren't any of those here. Both value and factor are evaluated only once and in order, and rem doesn't have any side effects.
This is not good Lisp though, because there's no reason to use a macro in this case. A function
(defun multp (value factor)
(zerop (rem value factor)))
is identical for all practical purposes. (Note the use of zerop. I think it makes things clearer in this case, but in cases where you need to highlight, that the value you're testing might still be meaningful if it's something other then zero, (= ... 0) might be better)
Your macro looks fine to me. I don't know what a leaky macro is, but yours is pretty straightforward and doesn't require any gensyms. As far as if this is "good" Lisp, my rule of thumb is to use a macro only when a function won't do, and in this case a function can be used in place of your macro. However, if this solution works for you there's no reason not to use it.
Well, in principle, a user could do this:
(flet ((= (&rest args) nil))
(multp 40 10))
which would evaluate to NIL... except that ANSI CL makes it illegal to rebind most standard symbols, including CL:=, so you're on the safe side in this particular case.
In generial, of course, you should be aware of both referential untransparency (capturing identifiers from the context the macro is expanded in) and macro unhygiene (leaking identifiers to expanded code).
No, no symbol introduced in the macro's "lexical closure" is released to the outside.
Note that leaking isn't NECESSARILY a bad thing, even if accidental leaking almost always is. For one project I worked on, I found that a macro similar to this was useful:
(defmacro ana-and (&rest forms)
(loop for form in (reverse forms)
for completion = form then `(let ((it ,form))
(when it
,completion))
finally (return completion)))
This allowed me to get "short-circuiting" of things needed to be done in sequence, with arguments carried over from previous calls in the sequence (and a failure signalled by returning NIL). The specific context this code is from is for a hand-written parser for a configuration file that has a cobbled-together-enough syntax that writing a proper parser using a parser generator was more work than hand-rolling.
Related
In Lisp, a function's arguments are evaluated first before entering the function body. Macro arguments stay not evaluated.
But sometimes, one wants to inject code pieces stored in variables into a macro. This means evaluating the argument for the macro first, and then apply the macro-of-choice on this evaluated result.
One has to resort to
(eval `(macro ,arg))
To achieve this - but eval does not behave correctly in different environments.
The best thing would be, if one could do:
(apply macro (list arg))
or
(funcall macro arg)
But since the macro is not a function this doesn't work.
Is it possible to achieve something like this? - To circumvent that problem oder to make the macro available in the functions namespace?
Or am I missing some other ways to solve such problems?
I came to this question while trying to answer How to produce HTML from a list. but also in Generate TYPECASE with macro in common lisp, Evaluate arguments passed to a macro that generates functions in lisp, and How to convert a list to code/lambda in scheme?. But I always thought while answering them it would be good to have an apply or funcall-like function which can take macros.
It is not clear what you are trying to do, although it is almost certain that you are confused about something. In particular if you are calling eval inside macroexpansions then in almost all cases you are doing something both seriously wrong and seriously dangerous. I can't ever think of a case where I've wanted macros which expand to things including eval and I have written Lisp for a very very long time.
That being said, here is how you call the function associated with a macro, and why it is very seldom what you want to do.
Macros are simply functions whose domain and range is source code: they are compilers from a language to another language. It is perfectly possible to call the function associated with a macro, but what that function will return is source code, and what you will then need to do with that source code is evaluate it. If you want a function which deals with run-time data which is not source code, then you need that function, and you can't turn a macro into that function by some magic trick which seems to be what you want to do: that magic trick does not, and can not, exist.
So for instance if I have a macro
(defmacro with-x (&body forms)
`(let ((x 1))
,#forms))
Then I can call its macro function on a bit of source code:
> (funcall (macro-function 'with-x)
'(with-x (print "foo")) nil)
(let ((x 1)) (print "foo"))
But the result of this is another bit of source code: I need to compile or evaluate it, and nothing I can do will get around this.
Indeed in (almost?) all cases this is just the same as macroexpand-1):
> (macroexpand-1 '(with-x (print "foo")))
(let ((x 1)) (print "foo"))
t
And you can probably write macroexpand-1 in terms of macro-function:
(defun macroexpand-1/equivalent (form &optional (env nil))
(if (and (consp form)
(symbolp (first form))
(macro-function (first form)))
(values (funcall (macro-function (first form)) form env)
t)
(values form nil)))
So, if the result of calling a macro is source code, what do you do with that source code to get a result which is not source code? Well, you must evaluate it. And then, well, since the evaluator expands macros for you anyway, you might as well just write something like
(defun evaluate-with-x (code)
(funcall (compile nil `(lambda ()
(with-x ,#code)))))
So you didn't need to call the macro's function in any case. And this is not the magic trick which turns macros into functions dealing with data which is not source code: it is a terrible horror which is entirely made of exploding parts.
A concrete example: CL-WHO
It looks like this question might have its origins in this one and the underlying problem there is that that's not what CL-WHO does. In particular it is a confusion to think that something like CL-WHO is a tool for taking some kind of list and turning it into HTML. It's not: it's a tool for taking the source code of a language which is built on CL but includes a way of expressing HTML output mingled with CL code, and compiles it into CL code which will do the same thing. It happens to be the case that CL source code is expressed as lists & symbols, but CL-WHO isn't really about that: it's a compiler from, if you like, 'the CL-WHO language' to CL.
So, let's try the trick we tried above and see why it's a disaster:
(defun form->html/insane (form)
(funcall
(compile nil `(lambda ()
(with-html-output-to-string (,(make-symbol "O"))
,#form)))))
And you might, if you did not look at this too closely, think that this function does in fact do the magic trick:
> (form->html/insane '(:p ((:a :href "foo") "the foo")))
"<p></p><a href='foo'>the foo</a>"
But it doesn't. What happens if we call form->html/insane on this perfectly innocuous list:
(:p (uiop/run-program:run-program "rm -rf $HOME" :output t))
Hint: don't call form->html/insane on this list if you don't have very good backups.
CL-WHO is an implementation of a programming language which is a strict superset of CL: if you try to turn it into a function to turn lists into HTML you end up with something involving the same nuclear weapon you tinker with every time you call eval, except that nuclear weapon is hidden inside a locked cupboard where you can't see it. But it doesn't care about that: if you set it off it will still reduce everything within a few miles to radioactive ash and rubble.
So if you want a tool which will turn lists – lists which aren't source code – into HTML then write that tool. CL-WHO might have the guts of such a tool in its implemenentation, but you can't use it as it is.
And this is the same problem you face whenever you are trying to abuse macros this way: the result of calling a macro's function is Lisp source code, and to evaluate that source code you need eval or an equivalent of eval. And eval is not only not a terrible solution to almost any problem: it's also a nuclear weapon. There are, perhaps problems for which nuclear weapons are good solutions, but they are few and far between.
I love Racket's #;. I want to see it in every language that I ever use again. Can it be added to other Lisps via their macro systems? Or does the commenting character break the macro system's ability to read the code?
A sufficient answer will demonstrate a macro being built in any Lisp other than Racket that allows for a change in the commenting system. You need not actually implement Racket's #;, but I would like it if you do. Lisps with the least similarity to Racket, e.g. Clojure or any non-Scheme will be particularity nice to see.
#; isn't a macro, it's what Common lisp would call a readmacro: what it does is defined at read time, not later than that. Read macros which aim to completely suppress input are mildly perilous because there needs to be a way of saying 'read the following thing, but ignore it', and that's only possible if any other readmacros behave well: there's nothing to stop someone defining a readmacro which produces some side-effect even if reading is suppressed.
However, well-behaved readmacros (which includes all of the standard ones and the rest of the standard reader) in CL won't do that: they'll listen to whether reading is being suppressed, and behave accordingly.
CL allows you to do this as standard by using its conditionalisation on features, and in particular #+(or) <expr> will always skip <expr>.
But you can define your own: #; is not predefined so you can define it:
(set-dispatch-macro-character
#\# #\;
(lambda (stream char n)
(declare (ignore char))
(let ((*read-suppress* t))
(dotimes (i (or n 1) (values))
(read stream)))))
After this, or at least with a better tested version of this, then #; <expr> (or obviously #;<expr>) will read as whitespace, and #2; ... ... will skip two following expressions:
> (let ((x #;1 #2; 2 3 4)) x)
4
What you are looking for is #+(or) reader macro.
Since (or) evaluates to nil, the condition is always false the following form is never evaluated.
Say I have a hash table at runtime that has strings as keys. Can a macro have access to this information and build a let expression from it?
(define env (hash 'a 123 'b 321))
(magic-let env (+ a b)) ; 444
I know I can hack around with with identifier-binding by replacing non-defined identifiers with a lookup in the hash table but then shadowing will not work as in a normal let.
Tagging scheme too as I assume its macro system is similar.
No, you can’t do that. At least not the way you describe.
The general reason why you cannot access runtime values within macros is simple: macros are fully expanded at compile time. When your program is compiled, the runtime values simply do not exist. A program can be compiled, and the bytecode can be placed on another computer, which will run it weeks later. Macro-expansion has already happened. No matter what happens at runtime, the program isn’t going to change.
This guarantee turns out to be incredibly important for a multitude of reasons, but that’s too general a discussion for this question. It would be relevant to discuss a particular question, which is why bindings themselves need to be static.
In Racket, as long as you are within a module (i.e. not at the top-level/REPL), all bindings can be resolved statically, at compile-time. This is a very useful property in other programming languages, mostly because the compiler can generate much more efficiently optimized code, but it is especially important in Racket or Scheme. This is because of how the macro system operates: in a language with hygienic macros, scope is complicated.
This is actually a very good thing—it is robust enough to support very complex systems that would be much harder to manage without hygiene—but it introduces some constraints:
Since every binding can be a macro or a runtime value, the binding needs to be known ahead of time in order to perform program expansion. The compiler needs to know if it needs to perform macro expansion or simply emit a variable reference.
Additionally, scoping rules are much more intricate because macro-introduced bindings live in their own scope. Because of this, binding scopes do not need to be strictly lexical.
Your magic-let could not work quite as you describe because the compiler could not possibly deduce the bindings for a and b statically. However, all is not lost: you could hook into #%top, a magical identifier introduced by the expander when encountering an unbound identifier. You could use this to replace unbound values with a hash lookup, and you could use syntax parameters to adjust #%top hygienically within each magic-let. Here’s an example:
#lang racket
(require (rename-in racket/base [#%top base-#%top])
racket/stxparam)
(define-syntax-parameter #%top (make-rename-transformer #'base-#%top))
(define-syntax-rule (magic-let env-expr expr ...)
(let ([env env-expr])
(syntax-parameterize ([#%top (syntax-rules ()
[(_ . id) (hash-ref env 'id)])])
(let () expr ...))))
(magic-let (hash 'a 123 'b 321) (+ a b)) ; => 444
Of course, keep in mind that this would replace all unbound identifiers with hash lookups. The effects of this are twofold. First of all, it will not shadow identifiers that are already bound:
(let ([a 1])
(magic-let (hash 'a 2)
a)) ; => 1
This is probably for the best, just to keep things semi-sane. It also means that the following would raise a runtime exception, not a compile-time error:
(magic-let (hash 'a 123) (+ a b))
; hash-ref: no value found for key
; key: 'b
I wouldn’t recommend doing this, as it goes against a lot of the Racket philosophy, and it would likely cause some hard-to-find bugs. There’s probably a better way to solve your problem without abusing things like #%top. Still, it is possible, if you really want it.
I have a question concerning evaluation of lists in lisp.
Why is (a) and (+ a 1) not evaluated,
(defun test (a) (+ a 1))
just like (print 4) is not evaluated here
(if (< 1 2) (print 3) (print 4))
but (print (+ 2 3)) is evaluated here
(test (print (+ 2 3)))
Does it have something to do with them being standard library functions? Is it possible for me to define functions like that in my lisp program?
As you probably know, Lisp compound forms are generally processed from the outside in. You must look at the symbol in the first position of the outermost nesting to understand a form. That symbol completely determines the meaning of the form. The following expressions all contain (b c) with completely different meaning; therefore, we cannot understand them by analyzing the (b c) part first:
;; Common Lisp: define a class A derived from B and C
(defclass a (b c) ())
;; Common Lisp: define a function of two arguments
(defun a (b c) ())
;; add A to the result of calling function B on variable C:
(+ a (b c))
Traditionally, Lisp dialects have divided forms into operator forms and function call forms. An operator form has a completely arbitrary meaning, determined by the piece of code which compiles or interprets that functions (e.g. the evaluation simply recurses over all of the function call's argument forms, and the resulting values are passed to the function).
From the early history, Lisp has allowed users to write their own operators. There existed two approaches to this: interpretive operators (historically known as fexprs) and compiling operators known as macros. Both hinge around the idea of a function which receives the unevaluated form as an argument, so that it can implement a custom strategy, thereby extending the evaluation model with new behaviors.
A fexpr type operator is simply handed the form at run-time, along with an environment object with which it can look up the values of variables and such. That operator then walks the form and implements the behavior.
A macro operator is handed the form at macro-expansion time (which usually happens when top-level forms are read, just before they are evaluated or compiled). Its job is not to interpret the form's behavior, but instead to translate it by generating code. I.e. a macro is a mini compiler. (Generated code can contain more macro calls; the macro expander will take care of that, ensuring that all macro calls are decimated.)
The fexpr approach fell out of favor, most likely because it is inefficient. It basically makes compilation impossible, whereas Lisp hackers valued compilation. (Lisp was already a compiled language from as early as circa 1960.) The fexpr approach is also hostile toward lexical environments; it requires the fexpr, which is a function, to be able to peer into the variable binding environment of the form in which its invoked, which is a kind of encapsulation violation that is not allowed by lexical scopes.
Macro writing is slightly more difficult, and in some ways less flexible than fexprs, but support for macro writing improved in Lisp through the 1960's into the 70's to make it close to as easy as possible. Macro originally had receive the whole form and then have to parse it themselves. The macro-defining system developed into something that provides macro functions with arguments that receive the broken-down syntax in easily digestible pieces, including some nested aspects of the syntax. The backquote syntax for writing code templates was also developed, making it much easier to express code generation.
So to answer your question, how can I write forms like that myself? For instance if:
;; Imitation of old-fashioned technique: receive the whole form,
;; extract parts from it and return the translation.
;; Common Lisp defmacro supports this via the &whole keyword
;; in macro lambda lists which lets us have access to the whole form.
;;
;; (Because we are using defmacro, we need to declare arguments "an co &optional al",
;; to make this a three argument macro with an optional third argument, but
;; we don't use those arguments. In ancient lisps, they would not appear:
;; a macro would be a one-argument function, and would have to check the number
;; of arguments itself, to flag bad syntax like (my-if 42) or (my-if).)
;;
(defmacro my-if (&whole if-form an co &optional al)
(let ((antecedent (second if-form)) ;; extract pieces ourselves
(consequent (third if-form)) ;; from whole (my-if ...) form
(alternative (fourth if-form)))
(list 'cond (list antecedent consequent) (list t alternative))))
;; "Modern" version. Use the parsed arguments, and also take advantage of
;; backquote syntax to write the COND with a syntax that looks like the code.
(defmacro my-if (antecedent consequent &optional alternative)
`(cond (,antecedent ,consequent) (t ,alternative))))
This is a fitting example because originally Lisp only had cond. There was no if in McCarthy's Lisp. That "syntactic sugar" was invented later, probably as a macro expanding to cond, just like my-if above.
if and defun are macros. Macros expand a form into a longer piece of code. At expansion time, none of the macro's arguments are evaluated.
When you try to write a function, but struggle because you need to implement a custom evaluation strategy, its a strong signal that you should be writing a macro instead.
Disclaimer: Depending on what kind of lisp you are using, if and defun might technically be called "special forms" and not macros, but the concept of delayed evaluation still applies.
Lisp consists of a model of evaluation of forms. Different Lisp dialects have different rules for those.
Let's look at Common Lisp.
data evaluates to itself
a function form is evaluated by calling the function on the evaluated arguments
special forms are evaluated according to rules defined for each special operator. The Common Lisp standard lists all of those, defines what they do in an informal way and there is no way to define new special operators by the user.
macros forms are transformed, the result is evaluated
How IF, DEFUN etc. works and what they evaluated, when it is doen and what is not evaluated is defined in the Common Lisp standard.
So... I'm new to scheme r6rs, and am learning macros. Can somebody explain to me what is meant by 'hygiene'?
Thanks in advance.
Hygiene is often used in the context of macros. A hygienic macro doesn't use variable names that can risk interfering with the code under expansion. Here is an example. Let's say we want to define the or special form with a macro. Intuitively,
(or a b c ... d) would expand to something like (let ((tmp a)) (if tmp a (or b c ... d))). (I am omitting the empty (or) case for simplicity.)
Now, if the name tmp was actually added in the code like in the above sketched expansion, it would be not hygienic, and bad because it might interfere with another variable with the same name. Say, we wanted to evaluate
(let ((tmp 1)) (or #f tmp))
Using our intuitive expansion, this would become
(let ((tmp 1)) (let ((tmp #f)) (if tmp (or tmp)))
The tmp from the macro shadows the outer-most tmp, and so the result is #f instead of 1.
Now, if the macro was hygienic (and in Scheme, it's automatically the case when using syntax-rules), then instead of using the name tmp for the expansion, you would use a symbol that is guaranteed not to appear anywhere else in the code. You can use gensym in Common Lisp.
Paul Graham's On Lisp has advanced material on macros.
If you imagine that a macro is simply expanded into the place where it is used, then you can also imagine that if you use a variable a in your macro, there might already be a variable a defined at the place where that macro is used.
This is not the a that you want!
A macro system in which something like this cannot happen, is called hygienic.
There are several ways to deal with this problem. One way is simply to use very long, very cryptic, very unpredictable variable names in your macros.
A slightly more refined version of this is the gensym approach used by some other macro systems: instead of you, the programmer coming up with a very long, very cryptic, very unpredictable variable name, you can call the gensym function which generates a very long, very cryptic, very unpredictable and unique variable name for you.
And like I said, in a hygienic macro system, such collisions cannot happen in the first place. How to make a macro system hygienic is an interesting question in itself, and the Scheme community has spent several decades on this question, and they keep coming up with better and better ways to do it.
I'm so glad to know that this language is still being used! Hygienic code is code that when injected (via a macro) does not cause conflicts with existing variables.
There is lots of good information on Wikipedia about this: http://en.wikipedia.org/wiki/Hygienic_macro
Here's what I found. Explaining what it means is another matter altogether!
http://www.r6rs.org/final/html/r6rs-lib/r6rs-lib-Z-H-1.html#node_toc_node_sec_12.1
Macros transform code: they take one bit of code and transform it into something else. As part of that transformation, they may surround that code with more code. If the original code references a variable a, and the code that's added around it defines a new version of a, then the original code won't work as expected because it will be accessing the wrong a: if
(myfunc a)
is the original code, which expects a to be an integer, and the macro takes X and transforms it to
(let ((a nil)) X)
Then the macro will work fine for
(myfunc b)
but (myfunc a) will get transformed to
(let ((a nil)) (myfunc a))
which won't work because myfunc will be applied to nil rather than the integer it is expecting.
A hygienic macro avoids this problem of the wrong variable getting accessed (and a similar problem the other way round), by ensuring that the names used are unique.
Wikipedia has a good explanation of hygienic macros.
Apart from all the things mentioned, there is one important other thing to Scheme's hygienic macros, which follow from the lexical scope.
Say we have:
(syntax-rules () ((_ a b) (+ a b)))
As part of a macro, surely it will insert the +, it will also insert it when there's a + already there, but then another symbol which has the same meaning as +. It binds symbols to the value they had in the lexical environment in which the syntax-rules lies, not where it is applied, we are lexically scoped after all. It will most likely insert a completely new symbol there, but one which is globally bound to the same meaning as + is at the place the macro is defined. This is most handy when we use a construct like:
(let ((+ *))
; piece of code that is transformed
)
The writer, or user of the macro thus needn't be occupied with ensuring its use goes well.