Overriding "this" identifier macro in Racket - macros

I made my own prototypal OO system in Racket which uses "this" for methods. Everything works great, but unfortunately it prevents Racket's native class system from using "this".
Here's what I want to do:
(require (only-in racket (this oldthis)))
(define-syntax this
(syntax-id-rules (set!)
((set! this ...) (void))
((this a ...) ((self) a ...))
(this (if (null? (self)) oldthis (self)))))
One thing I've tried to do is rename the above "this" macro to be "newthis" and make a helper macro as follows:
(define-syntax (this stx)
(syntax-case stx ()
(this #'(if (null? (self)) oldthis newthis))))
But in both cases Racket throws an error because I'm using "oldthis" outside of a class context. How do I merely reference the old "this" macro without expanding it and throwing an error? Or, more broadly, how can I make my "this" macro delegate to the old "this" macro when (self) is null?
Thanks in advance!
EDIT: Here is some code that uses "this" properly:
(define Foo (new))
(void (# Foo 'name "Foo")
(# Foo 'method (lambda () (# this 'name))))
You can then run Foo's "method" by calling
((# Foo 'method))
Which gives the correct result "Foo"
However, if I use Racket's class system:
(define foo%
(class object%
(super-new)
(define/public (displayself)
(displayln this))))
and run
(define foo (make-object foo%))
(send foo displayself)
I get (), always, since "this" returns (self) and (self) has in its closure a value "self" which is currently null. What I want to do is to check if (self) is null and, if it is, delegate to the built-in "this" macro.
EDIT2:
It looks like this answer is doing what I want to do. When I try it, however, I still get the same error about class context. I think the reason for that is that the #%top macro is invoked by (real-top . rest) and not by real-top on its own. In my case, however, oldthis gets invoked immediately and throws the error. I have no idea how to prevent that.
One thing I did that got close was to enclose oldthis in (eval 'oldthis), but that only worked in the current namespace. In order for that to work consistently, I would have to provide oldthis in conjunction with everything else, which is not only ugly but error prone, since an unsuspecting user of the library wouldn't necessarily understand that they have to pass oldthis all the time for anything to work properly. So I don't want to do it this way.
SOLVED:
(define-syntax this
(syntax-id-rules (set!)
((set! this ...) (void))
((this a ...) ((self) a ...))
(this (if (null? (self)) (eval 'this (module->namespace 'racket)) (self)))))
By providing the racket language as the namespace argument for eval, I can reference the old "this" without making any assumptions about the program using my OO library except that it have access to the racket language.

You should be able to do this directly with rename-in. Take the following code for example:
#lang racket
(require (rename-in racket [this old-this]))
(define this 42)
(define foo%
(class object%
(super-new)
(define/public (get-me)
old-this)))
(define bar (new foo%))
(send bar get-me)
Here we use rename-in to import old-this to point to the same binding as this in Racket. Which is needed because we immediately shadow that binding with: (define this 42)
Next, we define the foo% object, with one method: get-me, which just returns the current object with this (or specifically old-this.
Finally, we define a new foo% called bar, and call get-me. As you can see, it returns the bar object as you would expect.
If, instead of old-this, you returned this, then get-me would return 42, because of the definition. Likewise, if you removed the definition for this as 42, then this would still be bound to the one the class macro expects.

Related

Dynamically binding free identifiers in a function body

Consider a very simple actor language where an actor defines some local state and some methods that can be invoked by sending messages to the actor. In its implementation, one such method of the actor can be transformed into a function that defines the formal parameters of the method and accepts the current local state of the actor. Calling the method returns the new local state.
Binding the formal parameters in the body is no problem, but binding the local state seems to be more difficult. In the example at the end of the code below, in the body of the save method the a will remain unbound, despite an a (a different a) being bound by the generated evaluate-body function in the METHOD macro. The critical point in the code sample below is thus the METHOD macro, more specifically the evaluate-body function (which is where binding should happen, implying that my program design is reasonable)
Is there a way to hygienically bind this arbitrary set of free identifiers (currently only containing a, but it may be anything, really)?
#lang racket
(require (for-syntax syntax/parse))
(require racket/stxparam)
(struct actor (local-state methods))
(struct method (name formal-parameters body))
(define-syntax-parameter local-state-variables #f)
(define-syntax (ACTOR stx)
(syntax-parse stx
[(_ (LOCAL_STATE state-variable ...) method:expr ...+)
#'(syntax-parameterize ([local-state-variables '(state-variable ...)])
; For the sake of simplicity, an actor is currently a list of message handlers
(actor
(make-list (length '(state-variable ...)) (void))
(list method ...)))]))
(define-syntax (METHOD stx)
(syntax-parse stx
[(_ (name:id formal-parameter:id ...) body:expr ...+)
(with-syntax ([(local-state-variable ...) (syntax-parameter-value #'local-state-variables)])
#'(method
'name
'(formal-parameter ...)
(λ (formal-parameter ... #:local-state [current-state '()])
; the "a" that will be bound here is different from the free identifier "a" in the body
(define (evaluate-body local-state-variable ...)
body ...
(list local-state-variable ...))
(apply evaluate-body current-state))))]))
(ACTOR (LOCAL_STATE a)
(METHOD (save new-a)
; "a" is an unbound identifier
(set! a new-a)))
In order for the local state variables to have the proper lexical context, you need to store them as identifiers, not symbols. That is, in the result of the ACTOR macro, you need to change the syntax-parameterize to this:
#'(syntax-parameterize ([local-state-variables #'(state-variable ...)])
#| rest of the template (unchanged)... |#)
Note the replacement of quote/' with syntax/#'. This will store the identifiers with their lexical context instead of as symbols.
The next step is to properly introduce them within the METHOD macro. To do this, you just need to apply syntax-local-introduce to the value of the syntax parameter, which will add the macro introduction scope to the identifiers. You can also replace with-syntax with syntax-parse’s #:with clause to simplify things slightly, so the overall macro becomes this:
(define-syntax (METHOD stx)
(syntax-parse stx
[(_ (name:id formal-parameter:id ...) body:expr ...+)
#:with (local-state-variable ...)
(syntax-local-introduce (syntax-parameter-value #'local-state-variables))
#'(method #| rest of the template (unchanged)... |#)]))
This will work.
The reason syntax-local-introduce is needed here might be a little confusing, but the most intuitive way to think about it is by considering the “sets of scopes” hygiene model that Racket currently uses. In order for macro-introduced bindings to not conflict with user-defined bindings, each piece of syntax returned by a syntax transformer has a fresh scope attached to it, a scope that will never be attached to anything written by the user. Of course, some of the syntax in the result is syntax provided by the user, so the macroexpander needs to ensure it doesn’t attach the fresh scope to those syntax objects.
It’s not possible, in general, to figure out which syntax objects should be considered provided by the user since macro authors can “bend” hygiene and create new syntax objects from other ones. The solution, fortunately, is simple and elegant: just attach the macro introduction scope to all syntax objects provided by the user before handing them off to the macro, then flip the scopes on all pieces of syntax in the result. This way, the user-provided syntax objects will not have the macro introduction scope after the flipping occurs.
The syntax-local-introduce function lets you flip this special scope manually. In this case, since the value of local-state-variables should be treated like an input to the macro, but it isn’t automatically given the macro introduction scope by the macroexpander (since it isn’t a direct input to the macro), you have to add the scope yourself. That way, the macroexpander will remove the scope after the macro is expanded, and the identifier will end up with the proper lexical context.

How do I rename a procedure with macros in Racket?

Let's say I want to replace all occurrences of an procedure at compile time, for example all occurrences of cons with 😄. I tried two options that seemed natural:
1.
(define-syntax 😄
(syntax-rules ()
[(_ x y) (cons x y)]))
This works if I do something like (😄 2 3), but I can't use 😄 if it's not in application position, so I get a 'bad syntax' error if I do (map 😄 '(1 2) '(3 4)).
Then I thought to just replace the identifier itself:
2.
(define-syntax 😄
(λ (_) #'cons))
Now 😄 by itself gives me #<procedure:cons> and (map 😄 '(1 2) '(3 4)) gives the correct answer, but with (😄 2 3) the syntax transformer is taking all the arguments and replaces the whole expression with cons, which is not what I wanted.
How do I achieve what I want? Is there some kind of transformer that does this?
UPDATE: As I typed the last sentence, I invoked the "Rubber duck effect" and found make-rename-transformer which is the thing I want. However, the docs say "Such a transformer could be written manually", and it seems I have failed to do so with my 2 tries. How do you manually make such a transformer? (ignoring the bullet points in the docs for make-syntax-transformer)
Also, I know I can just use (define 😄 cons) but then that's a whole extra function call at runtime.
The absolute easiest (and probably best) way to do this would be to use rename-in to import an identifier under a different name:
(require (rename-in racket/base [cons 😄]))
This is also the most direct way to do it, since it won’t create a syntax transformer at all—it will create a new binding that is identical to cons in every way, including free-identifier=?. From the compiler’s perspective, that makes cons and 😄 indistinguishable.
However, this is a little bit magical. Another approach would be to manually create a rename transformer using make-rename-transformer:
(define-syntax 😄 (make-rename-transformer #'cons))
If you can’t use rename-in, using a rename transformer is the next best thing, since the free-identifier=? function recognizes rename transformers specially, so (free-identifier=? #'cons #'😄) will still be #t. This is useful for macros that care about syntax bindings, since 😄 will be accepted in all the same places cons would be.
Still, this is using some sort of primitive. If you really wanted to, how could you implement make-rename-transformer yourself? Well, the key here is that macro application can appear in two forms in Racket. If you have a macro foo, then it can be used in either of these ways:
foo
(foo ...)
The first case is an “id macro”, when a macro is used as a bare identifier, and the second is the more common case. If you want to write a macro that works just like an expression, though, you need to worry about both cases. You cannot do this with syntax-rules, which doesn’t allow id macros, but you can do this with syntax-id-rules:
(define-syntax 😄
(syntax-id-rules ()
[(_ . args) (cons . args)]
[_ cons]))
This pattern will make 😄 expand as expected in both scenarios. However, unlike the above two approaches, it will not cooperate with free-identifier=?, since 😄 is now a whole new binding bound to an ordinary macro from the macroexpander’s perspective.
From that point of view, is make-rename-transformer magical? Not really, but it is special in the sense that free-identifier=? handles it as a special case. If you wanted to, you could implement your own make-rename-transformer function and your own free-identifier=? function to get similar behavior:
(begin-for-syntax
(struct my-rename-transformer (id-stx)
#:property prop:procedure
(λ (self stx)
(with-syntax ([id (my-rename-transformer-id-stx self)])
((set!-transformer-procedure
(syntax-id-rules ()
[(_ . args) (id . args)]
[_ id]))
stx))))
(define (my-rename-target id-stx)
(let ([val (syntax-local-value id-stx (λ () #f))])
(if (my-rename-transformer? val)
(my-rename-target (my-rename-transformer-id-stx val))
id-stx)))
(define (my-free-identifier=? id-a id-b)
(free-identifier=? (my-rename-target id-a)
(my-rename-target id-b))))
This will allow you to do this:
(define-syntax 😄 (my-rename-transformer #'cons))
…and (my-free-identifier=? #'😄 #'cons) will be #t. However, it’s not recommended that you actually do this, for obvious reasons: not only is it unnecessary, it won’t really work, given that other macros won’t use my-free-identifier=?, just the ordinary free-identifier=?. For that reason, it’s highly recommended that you just use make-rename-transformer instead.
If you manually want to write such a macro, you need to use make-set!-transformer.
http://docs.racket-lang.org/reference/stxtrans.html?q=set-transformer#%28def.%28%28quote.~23~25kernel%29._make-set%21-transformer%29%29
Note that you need to handle both assignments (set! x e) and references x and applications (x arg0 arg1 ...) needs to be handled by the clauses in your syntax-case expression.
UPDATE
In Dybvig's book "The Scheme Programming Language" he has an example of a macro define-integrable that sounds to be just the thing you are after.
http://www.scheme.com/tspl4/syntax.html#./syntax:s61

trying to understand require in language extension

I'm trying to define a new language in racket, let's call it wibble. Wibble will allow modules to be loaded so it has to translate it's forms to Racket require forms. But I'm having trouble getting require to work when used in a language extension. I eventually tracked down my problems to the following strange behaviour.
Here's my reader which redefines read and read-syntax
=== wibble/lang/reader.rkt ===
#lang racket/base
(provide (rename-out (wibble-read read) (wibble-read-syntax read-syntax)))
(define (wibble-read in)
(wibble-read-syntax #f in))
(define (wibble-read-syntax src in)
#`(module #,(module-name src) wibble/lang
#,#(read-all src in)))
(define (module-name src)
(if (path? src)
(let-values (((base name dir?) (split-path src)))
(string->symbol (path->string (path-replace-suffix name #""))))
'anonymous-module))
(define (read-all src in)
(let loop ((all '()))
(let ((obj (read-syntax src in)))
(if (eof-object? obj)
(reverse all)
(loop (cons obj all))))))
and here's my much simplified language module, this introduces (require racket/base) into each wibble module
=== wibble/lang.rkt ===
#lang racket/base
(require (for-syntax racket/base))
(provide (rename-out (wibble-module-begin #%module-begin)) #%app #%datum #%top)
(define-syntax wibble-module-begin
(lambda (stx)
(syntax-case stx ()
((_ x ...) #`(#%module-begin (require #,(datum->syntax stx 'racket/base)) x ...)))))
With the above code then this wibble code 'works', i.e. there are no errors
#lang wibble
(cons 1 2)
(cons 3 4)
but the following
#lang wibble
(cons 1 2)
gives error message cons: unbound identifier in module in: cons
Really I'm just looking for an explanation as to what going on. I'm sure the difference is related to this from the racket docs (Racket Reference 3.1)
If a single form is provided, then it is partially expanded in a
module-begin context. If the expansion leads to #%plain-module-begin,
then the body of the #%plain-module-begin is the body of the module.
If partial expansion leads to any other primitive form, then the form
is wrapped with #%module-begin using the lexical context of the module
body; this identifier must be bound by the initial module-path import,
and its expansion must produce a #%plain-module-begin to supply the
module body. Finally, if multiple forms are provided, they are wrapped
with #%module-begin, as in the case where a single form does not
expand to #%plain-module-begin.
but even with that I don't understand why having a single form makes any difference, it's seems to be somthing to do with the timing of partial expansion but I'm not really sure. Nor do I understand why Racket treats a single form as a special case.
Incidentally I can fix the problem with a slight modification to my reader
(define (wibble-read-syntax src in)
#`(module #,(module-name src) wibble/lang
#,#(read-all src in) (void)))
Hard-coding a (void) form means I always have more than one form and eveything works.
Sorry for the long post, I'm just looking for some understanding of how this stuff works.
Alright, I think that I've figured it out.
Your intuition is correct in that the problem lies within the timing of the partial expansion of the single-form module body. Inside of your reader.rkt file, you produce a (module ...) form. As the quoted excerpt from your question states, the forms ... portion of this is then treated specially, since there is only one. Let's take a look at an excerpt from the documentation on partial expansion:
As a special case, when expansion would otherwise add an #%app, #%datum, or #%top identifier to an expression, and when the binding turns out to be the primitive #%app, #%datum, or #%top form, then expansion stops without adding the identifier.
I am almost certain that the partial expansion which occurs at this point does something to the cons identifier. This is the one part that I remain unsure of... my gut tells me that what's happening is that the partial expansion is attempting to find the binding for the cons identifier (since it is the first part of the parentheses, the identifier could be bound to a macro which should be expanded, so that needs to be checked) but is unable to, so it throws a tantrum. Note that even if cons has no phase 1 (syntax-expansion time) binding, the macro expander still expects there to be a phase 0 (runtime) binding for the identifier (among other things, this helps the expander remain hygienic). Because all of this partial expansion happens to the body of your (module ...) form (which is done before your (#%module-begin ...) form where you inject the (#%require ...) form), cons has no binding during the expansion, so the expansion, I believe, fails.
Nevertheless, a naive fix for your problem is to rewrite wibble-read-syntax as follows:
(define (wibble-read-syntax src in)
(let* ((read-in (read-all src in))
(in-stx (and (pair? read-in) (car read-in))))
#`(module #,(module-name src) wibble/lang
(require #,(datum->syntax in-stx 'racket/base))
#,#read-in))
You can then remove the (#%require ...) form from your (#%module-begin ...) macro.
That's not, in my opinion, the best way to fix the issue, however. As a matter of cleanliness, hard-coding in a require form like you've done in wibble/lang.rkt would make Eli Barzilay and co. cry. A much simpler way to do what you are trying to do is by updating your lang.rkt file to something like so:
=== wibble/lang.rkt ===
#lang racket/base
(require (for-syntax racket/base))
(provide (rename-out (wibble-module-begin #%module-begin))
(except-out (all-from-out racket/base) #%module-begin #%app #%datum #%top)
#%app #%datum #%top)
(define-syntax wibble-module-begin
(lambda (stx)
(syntax-case stx ()
((_ x ...) #`(#%module-begin x ...)))))
Writing in this convention removes the need for any hard-coded (require ...) forms and prevents subtle bugs like the one you've unearthed from occuring. If you are confused why this works, remember that you've already provided the #%module-begin identifier using this file, which is subsequently bound in all #lang wibble files. In principle, there is no limit on what identifiers you can bind in this fashion. If you would like some further reading, here's a shameless self-advertisement for a blog post I wrote a little while back on the subject.
I hope I've helped.
The problem is with the require (though I'm not sure I 100% understand all the behavior).
(require X) imports bindings from X with the lexical context of #'X. #'X here has the context of stx, which is the entire #'(module-begin x ...), which is not the context you want. You want the context of one of the cons expressions, i.e., one of the #'xs.
Something like this should work:
(define-syntax wibble-module-begin
(lambda (stx)
(syntax-case stx ()
[(_) #'(#%module-begin)]
[(m x y ...)
#`(#%module-begin
(require #,(datum->syntax #'x 'racket/base))
x y ...)])))
Though, as #belph warned, there's probably a more idiomatic way to accomplish what you want.
The behavior of your original program, and as you intuited, likely has to do with module's different treatment of single and multi sub-forms, but I think the "working" case might be an accident and could be a bug in the racket compiler.

Evaluate the arguments of a macro form

What is the best practice for selectively passing evaluated arguments to a macro form?
To elaborate: The usefulness of macros lies in its ability to receives unevaluated parameter, unlike the default evaluation rule for function forms. However, there is a legitimate use cases for evaluating macro arguments.
Consider a contrived example:
(defparameter *func-body* '((print i) (+ i 1)))
Suppose it would be nice that *func-body* could serve as the body of a macro our-defun that is defined as:
(defmacro our-defun (fun args &body body)
`(defun ,fun ,args ,#body))
So after (our-defun foo (i) (+ 1 i)), we could say (foo 1) to get 2. However, if we use (our-defun foo (i) *func-body*), the result of (foo 1) will be ((PRINT I) (+ I 1)) (i.e., the value of *func-body*). It would be nice if we can force the evaluation of *func-body* as an argument to the macro our-defun.
Currently, I can think of a technique of using compile and funcall to do this, as in
(funcall (compile nil `(lambda () (our-defun foo (i) ,#*func-body*))))
after which (our-defun 1) will print out 1 and return 2, as intended. I can think of case of making this work with eval, but I would rather stay away from eval due to its peculiarity in scoping.
This leads to my question at the begining, is there a more straightforward or native way to do this?
P.S.,
A not-so-contrived example is in the function (UPDATE-HOOK), which uses two library macros (ADD-HOOK) and (REMOVE-HOOK) and needs to evaluate its parameters. The (funcall (compile nil `(lambda () ...))) technique above is used here.
(defun update-hook (hook hook-name &optional code)
(funcall (compile nil `(lambda () (remove-hook ,hook ',hook-name))))
(unless (null code)
(compile hook-name `(lambda () ,#code))
(funcall (compile nil `(lambda () (add-hook ,hook ',hook-name))))))
That's slightly confused. A macro does not receive unevaluated parameters.
A macro gets source code and creates source code from that. Remember also that source code in Lisp is actually provided as data. The macro creates code, which evaluates some forms and some not.
Macros need to work in a compiling system. Before runtime. During compile time. All the macro sees is source code and then it creates source code from that. Think of macros as code transformations, not about evaluating arguments or not.
It would be nice if we can force the evaluation of *func-body* as an argument to the macro our-defun
That is not very clean. In a compiled system, you would need to make sure that *func-body* actually has a useful binding and that it can be resolved at COMPILE TIME.
If you have a macro like DEFUN, it makes sense to have the source code static. If you want to insert some source code into a form, then it could make sense to do that at read time:
(defun foo (i) #.`(,#*foo*))
But that's code I usually would want to avoid.
two library macros (ADD-HOOK) and (REMOVE-HOOK) and needs to evaluate its parameters.
Why should ADD-HOOK and REMOVE-HOOK be macros? If you don't have a real reason, they simply should be functions. Already since they make reuse difficult.
If you want to make ADD-HOOK and REMOVE-HOOK macros for some reason, then UPDATE-HOOK usually should be a macro, too.
The list you are giving to your macro has the form
(Quote (...))
So the list you actually want is the CADR of the list you get.

lisp macro expand with partial eval

I have following code which confuse me now, I hope some can tell me the difference and how to fix this.
(defmacro tm(a)
`(concat ,(symbol-name a)))
(defun tf(a)
(list (quote concat) (symbol-name a)))
I just think they should be the same effect, but actually they seem not.
I try to following call:
CL-USER> (tf 'foo)
(CONCAT "FOO")
CL-USER> (tm 'foo)
value 'FOO is not of the expected type SYMBOL.
[Condition of type TYPE-ERROR]
So, what's the problem?
What i want is:
(tm 'foo) ==> (CONCAT "FOO")
The first problem is that 'foo is expanded by the reader to (quote foo), which is not a symbol, but a list. The macro tries to expand (tm (quote foo)). The list (quote foo) is passed as the parameter a to the macro expansion function, which tries to get its symbol-name. A list is not a valid argument for symbol-name. Therefore, your macro expansion fails.
The second problem is that while (tm foo) (note: no quote) does expand to (concat "FOO"), this form will then be executed by the REPL, so that this is also not the same as your tf function. This is not surprising, of course, because macros do different things than functions.
First, note that
`(concat ,(symbol-name a))
and
(list (quote concat) (symbol-name a))
do the exact same thing. They are equivalent pieces of code (backquote syntax isn't restricted to macro bodies!): Both construct a list whose first element is the symbol CONCAT and whose second element is the symbol name of whatever the variable A refers to.
Clearly, this only makes sense if A refers to a symbol, which, as Svante has pointed out, isn't the case in the macro call example.
You could, of course, extract the symbol from the list (QUOTE FOO), but that prevents you from calling the macro like this:
(let ((x 'foo))
(tm x))
which raises the question of why you would event want to force the user of the macro to explicitly quote the symbol where it needs to be a literal constant anyway.
Second, the way macros work is this: They take pieces of code (such as (QUOTE FOO)) as arguments and produce a new piece of code that, upon macroexpansion, (more or less) replaces the macro call in the source code. It is often useful to reuse macro arguments within the generated code by putting them where they are going to be evaluated later, such as in
(defmacro tm2 (a)
`(print (symbol-name ,a)))
Think about what this piece of code does and whether or not my let example above works now. That should get you on the right track.
Finally, a piece of advice: Avoid macros when a function will do. It will make life much easier for both the implementer and the user.