I have a macro that produces a function. I understand that the best practice is to use gensyms to ensure naming conflicts don't happen. So I'm wondering if I need to follow this principle for the parameter names of the function the macro will define?
Here's my macro for reference:
(defmacro defroute
[routename uri meths handler]
`(defn ~routename
[~'req]
(let [req-meth# (:request-method ~'req)
bad-meth# (nil? (some #(= req-meth# %) ~meths))
any-meth# (= ~meths [:any])]
(if (:uri ~'req)
(if (and (route-matches ~uri ~'req) (and bad-meth# (not any-meth#)))
(method-not-allowed req-meth# (get-allowed ~meths))
(let [params# (route-matches ~uri ~'req)]
(if (nil? params#)
~'req
(~handler (assoc ~'req :route-params params#)))))
~'req))))
As you can see, I'm not using a gensym currently for the req parameter. I had originally and then wondered if it was necessary. Anyway, thanks for reading.
In this context the use of req is relatively safe because it establishes a local scope. As a function parameter, it will shadow any existing bindings to the symbol req in the calling namespace without damaging them. The cost of this is that if anyone tried to use the name req for any of the other parameters, such as handler, they could be in for a bit of a surprise. This code is not wrong in my view, though it does volate the principle of least supprise in some contexts. I don't see any reason not to use an auto-gensym for req considering it's only the work of adding a couple #s.
Related
In Lisp, a function's arguments are evaluated first before entering the function body. Macro arguments stay not evaluated.
But sometimes, one wants to inject code pieces stored in variables into a macro. This means evaluating the argument for the macro first, and then apply the macro-of-choice on this evaluated result.
One has to resort to
(eval `(macro ,arg))
To achieve this - but eval does not behave correctly in different environments.
The best thing would be, if one could do:
(apply macro (list arg))
or
(funcall macro arg)
But since the macro is not a function this doesn't work.
Is it possible to achieve something like this? - To circumvent that problem oder to make the macro available in the functions namespace?
Or am I missing some other ways to solve such problems?
I came to this question while trying to answer How to produce HTML from a list. but also in Generate TYPECASE with macro in common lisp, Evaluate arguments passed to a macro that generates functions in lisp, and How to convert a list to code/lambda in scheme?. But I always thought while answering them it would be good to have an apply or funcall-like function which can take macros.
It is not clear what you are trying to do, although it is almost certain that you are confused about something. In particular if you are calling eval inside macroexpansions then in almost all cases you are doing something both seriously wrong and seriously dangerous. I can't ever think of a case where I've wanted macros which expand to things including eval and I have written Lisp for a very very long time.
That being said, here is how you call the function associated with a macro, and why it is very seldom what you want to do.
Macros are simply functions whose domain and range is source code: they are compilers from a language to another language. It is perfectly possible to call the function associated with a macro, but what that function will return is source code, and what you will then need to do with that source code is evaluate it. If you want a function which deals with run-time data which is not source code, then you need that function, and you can't turn a macro into that function by some magic trick which seems to be what you want to do: that magic trick does not, and can not, exist.
So for instance if I have a macro
(defmacro with-x (&body forms)
`(let ((x 1))
,#forms))
Then I can call its macro function on a bit of source code:
> (funcall (macro-function 'with-x)
'(with-x (print "foo")) nil)
(let ((x 1)) (print "foo"))
But the result of this is another bit of source code: I need to compile or evaluate it, and nothing I can do will get around this.
Indeed in (almost?) all cases this is just the same as macroexpand-1):
> (macroexpand-1 '(with-x (print "foo")))
(let ((x 1)) (print "foo"))
t
And you can probably write macroexpand-1 in terms of macro-function:
(defun macroexpand-1/equivalent (form &optional (env nil))
(if (and (consp form)
(symbolp (first form))
(macro-function (first form)))
(values (funcall (macro-function (first form)) form env)
t)
(values form nil)))
So, if the result of calling a macro is source code, what do you do with that source code to get a result which is not source code? Well, you must evaluate it. And then, well, since the evaluator expands macros for you anyway, you might as well just write something like
(defun evaluate-with-x (code)
(funcall (compile nil `(lambda ()
(with-x ,#code)))))
So you didn't need to call the macro's function in any case. And this is not the magic trick which turns macros into functions dealing with data which is not source code: it is a terrible horror which is entirely made of exploding parts.
A concrete example: CL-WHO
It looks like this question might have its origins in this one and the underlying problem there is that that's not what CL-WHO does. In particular it is a confusion to think that something like CL-WHO is a tool for taking some kind of list and turning it into HTML. It's not: it's a tool for taking the source code of a language which is built on CL but includes a way of expressing HTML output mingled with CL code, and compiles it into CL code which will do the same thing. It happens to be the case that CL source code is expressed as lists & symbols, but CL-WHO isn't really about that: it's a compiler from, if you like, 'the CL-WHO language' to CL.
So, let's try the trick we tried above and see why it's a disaster:
(defun form->html/insane (form)
(funcall
(compile nil `(lambda ()
(with-html-output-to-string (,(make-symbol "O"))
,#form)))))
And you might, if you did not look at this too closely, think that this function does in fact do the magic trick:
> (form->html/insane '(:p ((:a :href "foo") "the foo")))
"<p></p><a href='foo'>the foo</a>"
But it doesn't. What happens if we call form->html/insane on this perfectly innocuous list:
(:p (uiop/run-program:run-program "rm -rf $HOME" :output t))
Hint: don't call form->html/insane on this list if you don't have very good backups.
CL-WHO is an implementation of a programming language which is a strict superset of CL: if you try to turn it into a function to turn lists into HTML you end up with something involving the same nuclear weapon you tinker with every time you call eval, except that nuclear weapon is hidden inside a locked cupboard where you can't see it. But it doesn't care about that: if you set it off it will still reduce everything within a few miles to radioactive ash and rubble.
So if you want a tool which will turn lists – lists which aren't source code – into HTML then write that tool. CL-WHO might have the guts of such a tool in its implemenentation, but you can't use it as it is.
And this is the same problem you face whenever you are trying to abuse macros this way: the result of calling a macro's function is Lisp source code, and to evaluate that source code you need eval or an equivalent of eval. And eval is not only not a terrible solution to almost any problem: it's also a nuclear weapon. There are, perhaps problems for which nuclear weapons are good solutions, but they are few and far between.
I'm interested in knowing what &allow-other-keys is supposed to do. I'm getting shockingly few references to it and no real definitions when I consult all documentation I can get my hands on. Best I can figure out is that it's effectively an error handling bit of code that you can put after &key to suppress the checking of key arguments when the macro is being called. For reference, this is where I found it:
(defmacro macro-name ((passed-variable
&rest open-args
&key &allow-other-keys)
&body body)
...)
&allow-other-keys is well documented: it suppresses Keyword Argument Checking.
In your example, it makes (macro-name (foo :bar 1) ...) an acceptable invocation of macro-name.
Although I think sds's answer is the right one, it's worth perhaps giving an example what is probably the canonical use-case for &allow-other-keys:
(defun open-unsigned-byte (f &rest kws &key &allow-other-keys)
(apply #'open f :element-type 'unsigned-byte kws))
This function wants to let open have whatever keyword arguments it takes, possibly including non-standard ones, but wants to make sure that the element-type is unsigned-byte, and that all the additional arguments passed to open are keyword arguments. This works because in the case of duplicated keyword arguments (which are legal) the leftmost one wins.
Say I have a hash table at runtime that has strings as keys. Can a macro have access to this information and build a let expression from it?
(define env (hash 'a 123 'b 321))
(magic-let env (+ a b)) ; 444
I know I can hack around with with identifier-binding by replacing non-defined identifiers with a lookup in the hash table but then shadowing will not work as in a normal let.
Tagging scheme too as I assume its macro system is similar.
No, you can’t do that. At least not the way you describe.
The general reason why you cannot access runtime values within macros is simple: macros are fully expanded at compile time. When your program is compiled, the runtime values simply do not exist. A program can be compiled, and the bytecode can be placed on another computer, which will run it weeks later. Macro-expansion has already happened. No matter what happens at runtime, the program isn’t going to change.
This guarantee turns out to be incredibly important for a multitude of reasons, but that’s too general a discussion for this question. It would be relevant to discuss a particular question, which is why bindings themselves need to be static.
In Racket, as long as you are within a module (i.e. not at the top-level/REPL), all bindings can be resolved statically, at compile-time. This is a very useful property in other programming languages, mostly because the compiler can generate much more efficiently optimized code, but it is especially important in Racket or Scheme. This is because of how the macro system operates: in a language with hygienic macros, scope is complicated.
This is actually a very good thing—it is robust enough to support very complex systems that would be much harder to manage without hygiene—but it introduces some constraints:
Since every binding can be a macro or a runtime value, the binding needs to be known ahead of time in order to perform program expansion. The compiler needs to know if it needs to perform macro expansion or simply emit a variable reference.
Additionally, scoping rules are much more intricate because macro-introduced bindings live in their own scope. Because of this, binding scopes do not need to be strictly lexical.
Your magic-let could not work quite as you describe because the compiler could not possibly deduce the bindings for a and b statically. However, all is not lost: you could hook into #%top, a magical identifier introduced by the expander when encountering an unbound identifier. You could use this to replace unbound values with a hash lookup, and you could use syntax parameters to adjust #%top hygienically within each magic-let. Here’s an example:
#lang racket
(require (rename-in racket/base [#%top base-#%top])
racket/stxparam)
(define-syntax-parameter #%top (make-rename-transformer #'base-#%top))
(define-syntax-rule (magic-let env-expr expr ...)
(let ([env env-expr])
(syntax-parameterize ([#%top (syntax-rules ()
[(_ . id) (hash-ref env 'id)])])
(let () expr ...))))
(magic-let (hash 'a 123 'b 321) (+ a b)) ; => 444
Of course, keep in mind that this would replace all unbound identifiers with hash lookups. The effects of this are twofold. First of all, it will not shadow identifiers that are already bound:
(let ([a 1])
(magic-let (hash 'a 2)
a)) ; => 1
This is probably for the best, just to keep things semi-sane. It also means that the following would raise a runtime exception, not a compile-time error:
(magic-let (hash 'a 123) (+ a b))
; hash-ref: no value found for key
; key: 'b
I wouldn’t recommend doing this, as it goes against a lot of the Racket philosophy, and it would likely cause some hard-to-find bugs. There’s probably a better way to solve your problem without abusing things like #%top. Still, it is possible, if you really want it.
So... I'm new to scheme r6rs, and am learning macros. Can somebody explain to me what is meant by 'hygiene'?
Thanks in advance.
Hygiene is often used in the context of macros. A hygienic macro doesn't use variable names that can risk interfering with the code under expansion. Here is an example. Let's say we want to define the or special form with a macro. Intuitively,
(or a b c ... d) would expand to something like (let ((tmp a)) (if tmp a (or b c ... d))). (I am omitting the empty (or) case for simplicity.)
Now, if the name tmp was actually added in the code like in the above sketched expansion, it would be not hygienic, and bad because it might interfere with another variable with the same name. Say, we wanted to evaluate
(let ((tmp 1)) (or #f tmp))
Using our intuitive expansion, this would become
(let ((tmp 1)) (let ((tmp #f)) (if tmp (or tmp)))
The tmp from the macro shadows the outer-most tmp, and so the result is #f instead of 1.
Now, if the macro was hygienic (and in Scheme, it's automatically the case when using syntax-rules), then instead of using the name tmp for the expansion, you would use a symbol that is guaranteed not to appear anywhere else in the code. You can use gensym in Common Lisp.
Paul Graham's On Lisp has advanced material on macros.
If you imagine that a macro is simply expanded into the place where it is used, then you can also imagine that if you use a variable a in your macro, there might already be a variable a defined at the place where that macro is used.
This is not the a that you want!
A macro system in which something like this cannot happen, is called hygienic.
There are several ways to deal with this problem. One way is simply to use very long, very cryptic, very unpredictable variable names in your macros.
A slightly more refined version of this is the gensym approach used by some other macro systems: instead of you, the programmer coming up with a very long, very cryptic, very unpredictable variable name, you can call the gensym function which generates a very long, very cryptic, very unpredictable and unique variable name for you.
And like I said, in a hygienic macro system, such collisions cannot happen in the first place. How to make a macro system hygienic is an interesting question in itself, and the Scheme community has spent several decades on this question, and they keep coming up with better and better ways to do it.
I'm so glad to know that this language is still being used! Hygienic code is code that when injected (via a macro) does not cause conflicts with existing variables.
There is lots of good information on Wikipedia about this: http://en.wikipedia.org/wiki/Hygienic_macro
Here's what I found. Explaining what it means is another matter altogether!
http://www.r6rs.org/final/html/r6rs-lib/r6rs-lib-Z-H-1.html#node_toc_node_sec_12.1
Macros transform code: they take one bit of code and transform it into something else. As part of that transformation, they may surround that code with more code. If the original code references a variable a, and the code that's added around it defines a new version of a, then the original code won't work as expected because it will be accessing the wrong a: if
(myfunc a)
is the original code, which expects a to be an integer, and the macro takes X and transforms it to
(let ((a nil)) X)
Then the macro will work fine for
(myfunc b)
but (myfunc a) will get transformed to
(let ((a nil)) (myfunc a))
which won't work because myfunc will be applied to nil rather than the integer it is expecting.
A hygienic macro avoids this problem of the wrong variable getting accessed (and a similar problem the other way round), by ensuring that the names used are unique.
Wikipedia has a good explanation of hygienic macros.
Apart from all the things mentioned, there is one important other thing to Scheme's hygienic macros, which follow from the lexical scope.
Say we have:
(syntax-rules () ((_ a b) (+ a b)))
As part of a macro, surely it will insert the +, it will also insert it when there's a + already there, but then another symbol which has the same meaning as +. It binds symbols to the value they had in the lexical environment in which the syntax-rules lies, not where it is applied, we are lexically scoped after all. It will most likely insert a completely new symbol there, but one which is globally bound to the same meaning as + is at the place the macro is defined. This is most handy when we use a construct like:
(let ((+ *))
; piece of code that is transformed
)
The writer, or user of the macro thus needn't be occupied with ensuring its use goes well.
I've been working through Practical Common Lisp and as an exercise decided to write a macro to determine if a number is a multiple of another number:
(defmacro multp (value factor)
`(= (rem ,value ,factor) 0))
so that :
(multp 40 10)
evaluates to true whilst
(multp 40 13)
does not
The question is does this macro leak in some way? Also is this "good" Lisp? Is there already an existing function/macro that I could have used?
Siebel gives an extensive rundown (for simple cases anyway) of possible sources of leaks, and there aren't any of those here. Both value and factor are evaluated only once and in order, and rem doesn't have any side effects.
This is not good Lisp though, because there's no reason to use a macro in this case. A function
(defun multp (value factor)
(zerop (rem value factor)))
is identical for all practical purposes. (Note the use of zerop. I think it makes things clearer in this case, but in cases where you need to highlight, that the value you're testing might still be meaningful if it's something other then zero, (= ... 0) might be better)
Your macro looks fine to me. I don't know what a leaky macro is, but yours is pretty straightforward and doesn't require any gensyms. As far as if this is "good" Lisp, my rule of thumb is to use a macro only when a function won't do, and in this case a function can be used in place of your macro. However, if this solution works for you there's no reason not to use it.
Well, in principle, a user could do this:
(flet ((= (&rest args) nil))
(multp 40 10))
which would evaluate to NIL... except that ANSI CL makes it illegal to rebind most standard symbols, including CL:=, so you're on the safe side in this particular case.
In generial, of course, you should be aware of both referential untransparency (capturing identifiers from the context the macro is expanded in) and macro unhygiene (leaking identifiers to expanded code).
No, no symbol introduced in the macro's "lexical closure" is released to the outside.
Note that leaking isn't NECESSARILY a bad thing, even if accidental leaking almost always is. For one project I worked on, I found that a macro similar to this was useful:
(defmacro ana-and (&rest forms)
(loop for form in (reverse forms)
for completion = form then `(let ((it ,form))
(when it
,completion))
finally (return completion)))
This allowed me to get "short-circuiting" of things needed to be done in sequence, with arguments carried over from previous calls in the sequence (and a failure signalled by returning NIL). The specific context this code is from is for a hand-written parser for a configuration file that has a cobbled-together-enough syntax that writing a proper parser using a parser generator was more work than hand-rolling.