My son and I are learning Racket together and are building a very simple text-based adventure for use directly from the REPL. So, for example, the player can type (go 'north) or (take 'apple).
After getting some basic stuff working, my son thought that quoting the noun was a bit of a pain (strangely, the parens don't bother him!), and so we hacked around with macros for a bit and we did get something working but it required an explicit function and a corresponding macro e.g.
(define (do-take item) ...)
(define-syntax (take stx)
(define item (cadr (syntax->datum stx)))
(datum->syntax stx `(do-take ',item)))
I figured we could do better than this, so I read around a bit more and came up with this:
(require (for-syntax racket/syntax))
(define-syntax (define-verb stx)
(syntax-case stx ()
[(_ (verb noun) body-first body-rest ...)
(with-syntax ([verb-fun (format-id stx "do-~a" #'verb)])
#'(begin
(define-syntax-rule (verb noun) (verb-fun 'noun))
(define (verb-fun noun) body-first body-rest ...)))]))
So now, we can write (define-verb (take item) ...) and the player at the REPL can type (take apple).
My question is whether, given what we want to achieve, this is a reasonable approach or whether there a more simple / idiomatic way to achieve the same thing?
In general, the main thing I would recommend doing is using the syntax/parse library. It has more tools for parsing syntax. You can even use forms like define-syntax-parser to make your macro even more concise. Rewriting your code using syntax/parse (dropping that one line because it doesn't seem to be doing anything), your macro would look like this:
#lang racket
(require syntax/parse/define
(for-syntax syntax/parse racket/syntax))
(define-syntax-parser define-verb
[(_ (verb:id noun) body ...+)
(define/syntax-parse verb-fun (format-id stx "do-~a" #'verb))
#'(begin
(define-simple-macro (verb noun) (verb-fun 'noun))
(define (verb-fun noun) body ...))])
This gives you a few nice things above the example you gave:
the :id ensures that verb is a literal identifier, rather than an expression.
the ...+ means you only need to have one body pattern, rather than two.
Using define/syntax-parse means your code does not get more indented than with-syntax. (Although this one is a matter of preference.)
Related
I am using syntax transformers to define macros in Racket. I want to create some helper functions to help me manipulate the syntax. However, the functions I defined outside the syntax transformer are not available inside the syntax transformer. For example, in the following code
(define (my-function x) (+ x 1))
(define-syntax my-macro
(lambda (stx)
(datum->syntax stx (my-function (cadr (syntax->datum stx))))))
I got the error "my-function: reference to an unbound identifier at phase: 1; the transformer environment".
After some searching, I am able to write the following code so that my-function is available inside the syntax transformer.
(begin-for-syntax
(define (my-function x) (+ x 1)))
(provide (for-syntax my-function))
(define-syntax my-macro
(lambda (stx)
(datum->syntax stx (my-function (cadr (syntax->datum stx))))))
But the problem is, my-function is not available outside the syntax transformer this time. Sometimes I want to check those helper functions in ordinary code, so I need to be able to call it from both inside and outside the syntax transformer, just like the function cadr. How can I achieve that?
I know my question has something to do with Racket's syntax model, in particular the concept of "phase level", but I never really understand it. If you could provide some easy-to-follow tutorials explaining it I would even be more grateful.
A common way is to define your function that you want to share across phases in another (sub)module. Then, require it twice.
#lang racket
(module common racket
(provide my-function)
(define (my-function x) (+ x 1)))
(require 'common
(for-syntax 'common))
(define-syntax my-macro
(lambda (stx)
(datum->syntax stx (my-function (cadr (syntax->datum stx))))))
(my-function 1)
(my-macro 123)
While experimenting with racket's macros, I stumbled into a definition that wasn't at first obvious to me why it was rejected. The code is short and otherwise is probably useless, but is as follows:
#lang racket
(define-syntax (go stx)
(syntax-case stx ()
[(_ id)
#'(module mod racket
(define it id))]
))
(go 'dummy)
The complaint is quote: unbound identifier; also, no #%app syntax transformer...
If I manually inline (define it id) to (define it 'dummy) then it works.
I had a hunch that ' ie. quote of (go 'dummy) that is bound by #lang racket is not recognized as the same binding within the submodule mod even though syntactically it is the same sequence of letters. If I strip 'dummy of all lexical context by round tripping as follows:
(with-syntax ([ok (datum->syntax #f (syntax->datum #'id))])
below the pattern match (_ id) and replace definition of it with (define it ok) then all is good again.
#lang racket
(define-syntax (go stx)
(syntax-case stx ()
[(_ id)
(with-syntax ([ok (datum->syntax #f (syntax->datum #'id))])
#'(module mod racket
(define it ok)))]
))
(go 'dummy)
I presume that my dilemma was caused by the hygiene system. However, is there a more direct solution to convince racket compiler that these identifiers, ie. quote are really the same without this boilerplate?
The expression that you insert for id in:
(module mod racket
(define it id))
is going to be evaluated in the context of the module.
Therefore the syntactic context id id needs to be the same as
the context of the submodule.
You describe one way of removing existing context. Here is another:
#lang racket
(require (for-syntax racket/base))
(define-syntax (go stx)
(syntax-case stx ()
[(_ id)
(with-syntax ([id (syntax->datum #'id)])
#'(module mod racket
(provide it)
(define it id)))]))
(go 42)
(require (submod "." mod))
it
In most macros it is a good thing that context is preserved, so
having to "boiler plate" to remove it seems okay to me.
Of course, if you experience to much boiler plate then write
a macro that inserts the boiler plate for you :-)
I have the following code that composes a traditional macro eat which uses a quote ('(eating food)), with a make-rename-transformer macro that is supposed to transform lunch into sandwich. The full code is:
#lang racket
(require (for-syntax syntax/parse))
(define-syntax lunch (make-rename-transformer #'sandwich))
(define-syntax (eat stx)
(syntax-parse stx
[(_ food)
#''(eating food)]))
(eat lunch)
Because lunch is just a rename transfomer for sandwich, I would expect it to evaluate to (eat sandwich), and thus '(eating sandwich), but when I run it, I get:
'(eating lunch)
Which is not what I expected. Is there any way I can modify this so that rename transforms in quotes are followed? (As if I had used the list function rather than quote.)
The problem here is just an order of expansion. Unlike functions which, to a first approximation anyway, evaluate inside out. Macros by nature expand outside in, which is an intrinsic property of macro expansion in languages like Racket.
So, the problem is that the first step of macro expa
#lang racket
(define-syntax lunch (make-rename-transformer #'sandwich))
'(eating lunch)
At this point lunch is no longer an identifier, but just a datum, so the rename transformer won't apply here.
What you need to do is tell the macro expander to expand food before you place it in the quote and turn it into datum. Racket's macro expander contains this functionality, [local-expand], if you combine this with #:when (syntax/parse's version of with-syntax), you can get the macro to evaluate food first.
Putting it all together, your code looks like:
#lang racket
(require (for-syntax syntax/parse))
(define-syntax lunch (make-rename-transformer #'sandwich))
(define-syntax (eat stx)
(syntax-parse stx
[(_ food)
#:with expanded-food (local-expand #'food 'expression #f)
#''(eating expanded-food)]))
(eat lunch)
And when you run this, you get:
'(eating sandwich)
I'm trying to define a new language in racket, let's call it wibble. Wibble will allow modules to be loaded so it has to translate it's forms to Racket require forms. But I'm having trouble getting require to work when used in a language extension. I eventually tracked down my problems to the following strange behaviour.
Here's my reader which redefines read and read-syntax
=== wibble/lang/reader.rkt ===
#lang racket/base
(provide (rename-out (wibble-read read) (wibble-read-syntax read-syntax)))
(define (wibble-read in)
(wibble-read-syntax #f in))
(define (wibble-read-syntax src in)
#`(module #,(module-name src) wibble/lang
#,#(read-all src in)))
(define (module-name src)
(if (path? src)
(let-values (((base name dir?) (split-path src)))
(string->symbol (path->string (path-replace-suffix name #""))))
'anonymous-module))
(define (read-all src in)
(let loop ((all '()))
(let ((obj (read-syntax src in)))
(if (eof-object? obj)
(reverse all)
(loop (cons obj all))))))
and here's my much simplified language module, this introduces (require racket/base) into each wibble module
=== wibble/lang.rkt ===
#lang racket/base
(require (for-syntax racket/base))
(provide (rename-out (wibble-module-begin #%module-begin)) #%app #%datum #%top)
(define-syntax wibble-module-begin
(lambda (stx)
(syntax-case stx ()
((_ x ...) #`(#%module-begin (require #,(datum->syntax stx 'racket/base)) x ...)))))
With the above code then this wibble code 'works', i.e. there are no errors
#lang wibble
(cons 1 2)
(cons 3 4)
but the following
#lang wibble
(cons 1 2)
gives error message cons: unbound identifier in module in: cons
Really I'm just looking for an explanation as to what going on. I'm sure the difference is related to this from the racket docs (Racket Reference 3.1)
If a single form is provided, then it is partially expanded in a
module-begin context. If the expansion leads to #%plain-module-begin,
then the body of the #%plain-module-begin is the body of the module.
If partial expansion leads to any other primitive form, then the form
is wrapped with #%module-begin using the lexical context of the module
body; this identifier must be bound by the initial module-path import,
and its expansion must produce a #%plain-module-begin to supply the
module body. Finally, if multiple forms are provided, they are wrapped
with #%module-begin, as in the case where a single form does not
expand to #%plain-module-begin.
but even with that I don't understand why having a single form makes any difference, it's seems to be somthing to do with the timing of partial expansion but I'm not really sure. Nor do I understand why Racket treats a single form as a special case.
Incidentally I can fix the problem with a slight modification to my reader
(define (wibble-read-syntax src in)
#`(module #,(module-name src) wibble/lang
#,#(read-all src in) (void)))
Hard-coding a (void) form means I always have more than one form and eveything works.
Sorry for the long post, I'm just looking for some understanding of how this stuff works.
Alright, I think that I've figured it out.
Your intuition is correct in that the problem lies within the timing of the partial expansion of the single-form module body. Inside of your reader.rkt file, you produce a (module ...) form. As the quoted excerpt from your question states, the forms ... portion of this is then treated specially, since there is only one. Let's take a look at an excerpt from the documentation on partial expansion:
As a special case, when expansion would otherwise add an #%app, #%datum, or #%top identifier to an expression, and when the binding turns out to be the primitive #%app, #%datum, or #%top form, then expansion stops without adding the identifier.
I am almost certain that the partial expansion which occurs at this point does something to the cons identifier. This is the one part that I remain unsure of... my gut tells me that what's happening is that the partial expansion is attempting to find the binding for the cons identifier (since it is the first part of the parentheses, the identifier could be bound to a macro which should be expanded, so that needs to be checked) but is unable to, so it throws a tantrum. Note that even if cons has no phase 1 (syntax-expansion time) binding, the macro expander still expects there to be a phase 0 (runtime) binding for the identifier (among other things, this helps the expander remain hygienic). Because all of this partial expansion happens to the body of your (module ...) form (which is done before your (#%module-begin ...) form where you inject the (#%require ...) form), cons has no binding during the expansion, so the expansion, I believe, fails.
Nevertheless, a naive fix for your problem is to rewrite wibble-read-syntax as follows:
(define (wibble-read-syntax src in)
(let* ((read-in (read-all src in))
(in-stx (and (pair? read-in) (car read-in))))
#`(module #,(module-name src) wibble/lang
(require #,(datum->syntax in-stx 'racket/base))
#,#read-in))
You can then remove the (#%require ...) form from your (#%module-begin ...) macro.
That's not, in my opinion, the best way to fix the issue, however. As a matter of cleanliness, hard-coding in a require form like you've done in wibble/lang.rkt would make Eli Barzilay and co. cry. A much simpler way to do what you are trying to do is by updating your lang.rkt file to something like so:
=== wibble/lang.rkt ===
#lang racket/base
(require (for-syntax racket/base))
(provide (rename-out (wibble-module-begin #%module-begin))
(except-out (all-from-out racket/base) #%module-begin #%app #%datum #%top)
#%app #%datum #%top)
(define-syntax wibble-module-begin
(lambda (stx)
(syntax-case stx ()
((_ x ...) #`(#%module-begin x ...)))))
Writing in this convention removes the need for any hard-coded (require ...) forms and prevents subtle bugs like the one you've unearthed from occuring. If you are confused why this works, remember that you've already provided the #%module-begin identifier using this file, which is subsequently bound in all #lang wibble files. In principle, there is no limit on what identifiers you can bind in this fashion. If you would like some further reading, here's a shameless self-advertisement for a blog post I wrote a little while back on the subject.
I hope I've helped.
The problem is with the require (though I'm not sure I 100% understand all the behavior).
(require X) imports bindings from X with the lexical context of #'X. #'X here has the context of stx, which is the entire #'(module-begin x ...), which is not the context you want. You want the context of one of the cons expressions, i.e., one of the #'xs.
Something like this should work:
(define-syntax wibble-module-begin
(lambda (stx)
(syntax-case stx ()
[(_) #'(#%module-begin)]
[(m x y ...)
#`(#%module-begin
(require #,(datum->syntax #'x 'racket/base))
x y ...)])))
Though, as #belph warned, there's probably a more idiomatic way to accomplish what you want.
The behavior of your original program, and as you intuited, likely has to do with module's different treatment of single and multi sub-forms, but I think the "working" case might be an accident and could be a bug in the racket compiler.
What's the simplest way to define a capturing macro using define-syntax or define-syntax-rule in Racket?
As a concrete example, here's the trivial aif in a CL-style macro system.
(defmacro aif (test if-true &optional if-false)
`(let ((it ,test))
(if it ,if-true ,if-false)))
The idea is that it will be bound to the result of test in the if-true and if-false clauses. The naive transliteration (minus optional alternative) is
(define-syntax-rule (aif test if-true if-false)
(let ((it test))
(if it if-true if-false)))
which evaluates without complaint, but errors if you try to use it in the clauses:
> (aif "Something" (displayln it) (displayln "Nope")))
reference to undefined identifier: it
The anaphora egg implements aif as
(define-syntax aif
(ir-macro-transformer
(lambda (form inject compare?)
(let ((it (inject 'it)))
(let ((test (cadr form))
(consequent (caddr form))
(alternative (cdddr form)))
(if (null? alternative)
`(let ((,it ,test))
(if ,it ,consequent))
`(let ((,it ,test))
(if ,it ,consequent ,(car alternative)))))))))
but Racket doesn't seem to have ir-macro-transformer defined or documented.
Racket macros are designed to avoid capture by default. When you use define-syntax-rule it will respect lexical scope.
When you want to "break hygiene" intentionally, traditionally in Scheme you have to use syntax-case and (carefully) use datum->syntax.
But in Racket the easiest and safest way to do "anaphoric" macros is with a syntax parameter and the simple define-syntax-rule.
For example:
(require racket/stxparam)
(define-syntax-parameter it
(lambda (stx)
(raise-syntax-error (syntax-e stx) "can only be used inside aif")))
(define-syntax-rule (aif condition true-expr false-expr)
(let ([tmp condition])
(if tmp
(syntax-parameterize ([it (make-rename-transformer #'tmp)])
true-expr)
false-expr)))
I wrote about syntax parameters here and also you should read Eli Barzilay's Dirty Looking Hygiene blog post and Keeping it Clean with Syntax Parameters paper (PDF).
See Greg Hendershott's macro tutorial. This section uses anaphoric if as example:
http://www.greghendershott.com/fear-of-macros/Syntax_parameters.html
Although the answer above is the accepted way to implement aif in the Racket community, it has severe flaws. Specifically, you can shadow it by defining a local variable named it.
(let ((it 'gets-in-the-way))
(aif 'what-i-intended
(display it)))
The above would display gets-in-the-way instead of what-i-intended, even though aif is defining its own variable named it. The outer let form renders aif's inner let definition invisible. This is what the Scheme community wants to happen. In fact, they want you to write code that behaves like this so badly, that they voted to have my original answer deleted when I wouldn't concede that their way was better.
There is no bug-free way to write capturing macros in Scheme. The closest you can come is to walk down the syntax tree that may contain variables you want to capture and explicitly strip the scoping information that they contain, replacing it with new scoping information that forces them to refer to your local versions of those variables. I wrote three "for-syntax" functions and a macro to help with this:
(begin-for-syntax
(define (contains? atom stx-list)
(syntax-case stx-list ()
(() #f)
((var . rest-vars)
(if (eq? (syntax->datum #'var)
(syntax->datum atom))
#t
(contains? atom #'rest-vars)))))
(define (strip stx vars hd)
(if (contains? hd vars)
(datum->syntax stx
(syntax->datum hd))
hd))
(define (capture stx vars body)
(syntax-case body ()
(() #'())
(((subform . tl) . rest)
#`(#,(capture stx vars #'(subform . tl)) . #,(capture stx vars #'rest)))
((hd . tl)
#`(#,(strip stx vars #'hd) . #,(capture stx vars #'tl)))
(tl (strip stx vars #'tl)))))
(define-syntax capture-vars
(λ (stx)
(syntax-case stx ()
((_ (vars ...) . body)
#`(begin . #,(capture #'(vars ...) #'(vars ...) #'body))))))
That gives you the capture-vars macro, which allows you to explicitly name the variables from the body you'd like to capture. aif can then be written like this:
(define-syntax aif
(syntax-rules ()
((_ something true false)
(capture-vars (it)
(let ((it something))
(if it true false))))
((_ something true)
(aif something true (void)))))
Note that the aif I have defined works like regular Scheme's if in that the else-clause is optional.
Unlike the answer above, it is truly captured. It's not merely a global variable:
(let ((it 'gets-in-the-way))
(aif 'what-i-intended
(display it)))
The inadequacy of just using a single call to datum->syntax
Some people think that all you have to do to create a capturing macro is use datum->syntax on one of the top forms passed to your macro, like this:
(define-syntax aif
(λ (stx)
(syntax-case stx ()
((_ expr true-expr false-expr)
(with-syntax
((it (datum->syntax #'expr 'it)))
#'(let ((it expr))
(if it true-expr false-expr))))
((_ expr true-expr)
#'(aif expr true-expr (void))))))
Just using datum->syntax is only a 90% solution to writing capturing macros. It will work in most cases, but break in some cases, specifically if you incorporate a capturing macro written this way in another macro. The above macro will only capture it if the expr comes from the same scope as the true-expr. If they come from different scopes (this can happen merely by wrapping the user's expr in a form generated by your macro), then the it in true-expr will not be captured and you'll be left asking yourself "WTF won't it capture?"
You may be tempted to quick-fix this by using (datum->syntax #'true-expr 'it) instead of (datum->syntax #'expr 'it). In fact this makes the problem worse, since now you won't be able to use aif to define acond:
(define-syntax acond
(syntax-rules (else)
((_) (void))
((_ (condition . body) (else . else-body))
(aif condition (begin . body) (begin . else-body)))
((_ (condition . body) . rest)
(aif condition (begin . body) (acond . rest)))))
If aif is defined using the capture-vars macro, the above will work as expected. But if it's defined by using datum->syntax on the true-expr, the the addition of begin to the bodies will result in it being visible in the scope of acond's macro definition instead of the code that invoked acond.
The impossibility of really writing a capturing macro in Racket
This example was brought to my attention, and demonstrates why you just can't write a real capturing macro in Scheme:
(define-syntax alias-it
(syntax-rules ()
((_ new-it . body)
(let ((it new-it)) . body))))
(aif (+ 1 2) (alias-it foo ...))
capture-vars cannot capture the it in alias-it's macroexpansion, because it won't be on the AST until after aif is finished expanding.
It is not possible at all to fix this problem, because the macro definition of alias-it is most probably not visible from the scope of aif's macro definition. So when you attempt to expand it within aif, perhaps by using expand, alias-it will be treated as a function. Testing shows that the lexical information attached to alias-it does not cause it to be recognized as a macro for a macro definition written out of scope from alias-it.
Some would argue that this shows why the syntax-parameter solution is the superior solution, but perhaps what it really shows is why writing your code in Common Lisp is the superior solution.