I currently have a bunch of splicing syntax classes I use to generate code.
They look like this:
(define-splicing-syntax-class vec-exp
(pattern (~seq x y)
#:with result #'(vec x y)))
The goal is to be able to match a sequence x y anywhere and replace it with (vec x y).
The only way I see for now is by creating an attribute called result and use it:
> (syntax-parse #'(position 4.2 5.7)
[(<name> <pos>:vec-exp)
(attribute <pos>.result)])
#'(vec 4.2 5.7)
Is there a way to change my code so that I can get the same result by writing the following?
> (syntax-parse #'(position 4.2 5.7)
[(<name> <pos>:vec-exp)
(attribute <pos>)])
#'(4.2 5.7) ;; not what I want
FWIW, you can do this. Not sure if it's acceptable for you or not.
(require syntax/parse
(for-syntax syntax/parse))
(define-splicing-syntax-class vec-exp
(pattern (~seq x y) #:with result #'(vec x y)))
(define-syntax ~res
(pattern-expander
(syntax-parser
[(_ pat cls)
#'(~and (~var PAT cls) (~bind [pat (attribute PAT.result)]))])))
And then:
> (syntax-parse #'(position 4.2 5.7)
[(<name> (~res <pos> vec-exp))
(attribute <pos>)])
#'(vec 4.2 5.7)
I don't think so. The pattern (<name> <pos>:vec-exp) means "The input must be a list; bind its elements to pattern variables <name> and <pos>." Those pattern variables provide access to what got matched. The attributes returned by the syntax class are what got generated. The syntax-parse system is very fastidious about keeping those two concepts distinct, so I don't think it will let you replace one with the other.
Are you trying to make your macro more readable or less error-prone? If so, maybe tell us a little more. Maybe there is a way to do that.
Related
make-variable-transformer (or make-set!-transformer, as it is called in Racket) can work with identifiers hardcoded in the literal list of the macro definition. The following example with set! invariably comes up when variable transformers are mentioned:
(make-variable-transformer
(λ (stx)
(syntax-case stx (set!)
((set! id _) ...)
(id ...))))
This is nice and all, useful for transparently integrating foreign structures with primitive operations known ahead of time, and it's a bonus that it can work through identifier syntax and rename transformers.
But what I'm wondering is if it's possible to work with syntax dynamically like:
(let-syntax ((# (make-variable-transformer
(λ (stx)
(syntax-case stx ()
((v # i) (vector? #'v) #'(vector-ref v i)))))))
(#(0 1 2) # 1))
=> 1
This doesn't work because the macro call doesn't match the template as syntax-case expects # to be in the initial position since there's no v in the literal list (and it probably assigns # to v pattern variable).
In short: Is it possible to write a syntax transformer that can accomplish this without reader extensions or overriding application, perhaps through a metamacro that rewrites the literal token list of an inner syntax-case (à la Petrofsky extraction)?
NB: The vector example itself is unimportant and I'm not interested in alternative solutions to this exact use-case.
since there's no v in the literal list (and it probably assigns # to v pattern variable).
Not really. set! is a special case that the macro expander handles specifically to make it cooperate with make-variable-transformer. But for other literals, they will fail. E.g.,
(let-syntax ((# (make-variable-transformer
(λ (stx)
(syntax-case stx (v)
((v # i) #'1))))))
(v # 1))
fails with v: unbound identifier.
The second issue with your above code is the side condition (vector? #'v). #'v is a syntax object, so (vector? #'v) will always result in #f. It's unclear what is the right behavior. For example, do you intend for:
(define v (vector 1 2 3))
(v # 1)
to work? If so, a compile-time side condition would be inappropriate, because it's not known if v is a vector at compile-time.
For your main question, the answer is no. It's not possible under the constraints that you imposed. The expansion steps are detailed here, and none of the steps looks beyond the head of the list.
But if we don't care about the constraints. I.e., overriding #%app is OK. It could work.
An issue that you need to think about is, suppose you have (a b c) where b is your kind of macro and a is a regular macro. Who should get the control first? If a should get the control first, you can override #%app to implement this kind of macro. Here's my quick implementation.
#lang racket
(require syntax/parse/define
(only-in racket [#%app racket:#%app])
(for-syntax syntax/apply-transformer))
(begin-for-syntax
(struct my-transformer (t)))
(define-syntax-parser #%app
[(_ x ...)
(define transformer
(for/first ([operand (attribute x)]
#:when (and (identifier? operand)
(my-transformer?
(syntax-local-value operand (λ () #f)))))
(syntax-local-value operand)))
(cond
[transformer (local-apply-transformer
(my-transformer-t transformer)
#'(x ...)
'expression)]
[else #'(racket:#%app x ...)])])
(define-syntax #
(my-transformer
(syntax-parser
[(v _ i) #'(vector-ref v i)])))
(define v (vector 42 1337 1729))
(v # 1) ;=> 1337
Finally, you can always override #%module-begin and simulate the macro expander. It's an overkill solution, but could be appropriate if you want more advanced features, like allowing users to customize precedence so that b is expanded before a.
I'm in a process of implementing Hygienic macros in my Scheme implementation, I've just implemented syntax-rules, but I have this code:
(define odd?
(syntax-rules ()
((_ x) (not (even? x)))))
what should be the difference between that and this:
(define-syntax odd?
(syntax-rules ()
((_ x) (not (even? x)))))
from what I understand syntax-rules just return syntax transformer, why you can't just use define to assign that to symbol? Why I need to use define-syntax? What extra stuff that expression do?
Should first also work in scheme? Or only the second one?
Also what is the difference between let vs let-syntax and letrec vs letrec-syntax. Should (define|let|letrec)-syntax just typecheck if the value is syntax transformer?
EDIT:
I have this implementation, still using lisp macros:
;; -----------------------------------------------------------------------------
(define-macro (let-syntax vars . body)
`(let ,vars
,#(map (lambda (rule)
`(typecheck "let-syntax" ,(car rule) "syntax"))
vars)
,#body))
;; -----------------------------------------------------------------------------
(define-macro (letrec-syntax vars . body)
`(letrec ,vars
,#(map (lambda (rule)
`(typecheck "letrec-syntax" ,(car rule) "syntax"))
vars)
,#body))
;; -----------------------------------------------------------------------------
(define-macro (define-syntax name expr)
(let ((expr-name (gensym)))
`(define ,name
(let ((,expr-name ,expr))
(typecheck "define-syntax" ,expr-name "syntax")
,expr-name))))
This this code correct?
Should this code works?
(let ((let (lambda (x) x)))
(let-syntax ((odd? (syntax-rules ()
((_ x) (not (even? x))))))
(odd? 11)))
This question seems to imply some deep confusion about macros.
Let's imagine a language where syntax-rules returns some syntax transformer function (I am not sure this has to be true in RnRS Scheme, it is true in Racket I think), and where let and let-syntax were the same.
So let's write this function:
(define (f v)
(let ([g v])
(g e (i 10)
(if (= i 0)
i
(e (- i 1))))))
Which we can turn into this, of course:
(define (f v n)
(v e (i n)
(if (<= i 0)
i
(e (- i 1)))))
And I will tell you in addition that there is no binding for e or i in the environment.
What is the interpreter meant to do with this definition? Could it compile it? Could it safely infer that i can't possibly make any sense since it is used as a function and then as a number? Can it safely do anything at all?
The answer is that no, it can't. Until it knows what the argument to the function is it can't do anything. And this means that each time f is called it has to make that decision again. In particular, v might be:
(syntax-rules ()
[(_ name (var init) form ...)
(letrec ([name (λ (var)
form ...)])
(name init))]))
Under which the definition of f does make some kind of sense.
And things get worse: much worse. How about this?
(define (f v1 v2 n)
(let ([v v1])
(v e (i n)
...
(set! v (if (eq? v v1) v2 v1))
...)))
What this means is that a system like this wouldn't know what the code it was meant to interpret meant until, the moment it was interpreting it, or even after that point, as you can see from the second function above.
So instead of this horror, Lisps do something sane: they divide the process of evaluating bits of code into phases where each phase happens, conceptually, before the next one.
Here's a sequence for some imagined Lisp (this is kind of close to what CL does, since most of my knowledge is of that, but it is not intended to represent any particular system):
there's a phase where the code is turned from some sequence of characters to some object, possibly with the assistance of user-defined code;
there's a phase where that object is rewritten into some other object by user- and system-defined code (macros) – the result of this phase is something which is expressed in terms of functions and some small number of primitive special things, traditionally called 'special forms' which are known to the processes of stage 3 and 4;
there may be a phase where the object from phase 2 is compiled, and that phase may involve another set of user-defined macros (compiler macros);
there is a phase where the resulting code is evaluated.
And for each unit of code these phases happen in order, each phase completes before the next one begins.
This means that each phase in which the user can intervene needs its own set of defining and binding forms: it needs to be possible to say that 'this thing controls what happens at phase 2' for instance.
That's what define-syntax, let-syntax &c do: they say that 'these bindings and definitions control what happens at phase 2'. You can't, for instance, use define or let to do that, because at phase 2, these operations don't yet have meaning: they gain meaning (possibly by themselves being macros which expand to some primitive thing) only at phase 3. At phase 2 they are just bits of syntax which the macro is ingesting and spitting out.
I'd like to replace all occurrences of v1 in a syntax object like #'(or (and v1 v2) (and v1 v3)) to v4 to get #'(or (and v4 v2) (and v4 v3)). What would be the easiest way to do so in Racket? Should I convert the syntax to list or string to replace and convert it back to syntax?
There are several different strategies you can use depending on what the final syntax object is used for. In particular, it depends on whether you can expand the syntax to get a different syntax object with the same behavior, or whether you have to leave everything exactly as it was.
1. If the final syntax object is used as an expression in the output of a macro
If the final syntax object is only used as an expression in the output of a macro, then expanding the syntax object is fine, since the run-time behavior is what matters, not the exact form of the syntax. In this case, you can expand the syntax object in an internal-definition-context that includes the substitution.
;; create a context where x-old is renamed to x-new
(define ctx (syntax-local-make-definition-context))
(syntax-local-bind-syntaxes
(list x-old)
#`(make-rename-transformer (quote-syntax #,x-new))
ctx)
;; expand the syntax in that context
(local-expand stx 'expression '() ctx)
2. If the final syntax object is supposed to be kept exactly as is, and cannot be expanded
If the final syntax object is supposed to be kept exactly as is, except for the substitution, then you can't expand it. You have to traverse it somehow to do the substitution. This comes with several problems if the code you're substituting into might use certain features like quote or syntax->datum. However, there are times when its necessary, and for those times I use a traverse-stx/recur function with this signature:
;; traverse-stx/recur : Stx [Stx -> Stx] -> Stx
;; Traverses `stx`, calling the `recur` function on every sub-piece
(define (traverse-stx/recur stx recur)
....)
Which I use like this:
;; stx-subst : Stx Id Id -> Stx
;; Replaces every instance of `x-old` with `x-new` in the syntax `stx`
(define (stx-subst stx x-old x-new)
;; traverse : Stx -> Stx
(define (traverse s)
(cond [(and (identifier? stx) (free-identifier=? stx x-old))
x-new]
[else
;; pass "yourself" as the recur callback, so that it calls
;; you on every sub-piece
(traverse-stx/recur stx traverse)]))
(traverse s))
The definition of traverse-stx/recur might depend on the language you're traversing, but if it's literally just arbitrary s-expressions with no "meaning" that you're afraid of changing, then it can be structured just like a normal s-expression traversal, though with stx-null?, stx-car, stx-cdr, etc. (from the syntax/stx library) instead of the normal null?, car, cdr, etc.
NOTE: however you define the traversal for your language, a helper function like this might be useful:
;; restore : Stx Any -> Stx
;; Any Any -> Any
(define (restore orig datum)
(if (syntax? orig) (datum->syntax orig datum orig orig) datum))
3. When you need to rely on and preserve the "meaning" in a different core language
In some rare cases, might want to expand to a different "core language" than Racket's core forms. This is still an active area of research and hasn't been totally figured out yet. However, current strategies involve traversing the syntax object manually like (2), while also expanding the syntax using an internal-definition-context like (1), and reconstructing the syntax after the expansion.
The best explanation I've seen so far for how to do this is in this Blog Post by Alexis King. But this is very hard to do correctly, and the more complicated your "core language" is, the harder it becomes.
Use with-syntax.
(with-syntax ([v1 #'v4])
#'(or (and v1 v2) (and v1 v3)))
The output:
#<syntax:3:4 (or (and v4 v2) (and v4 v3))>
Turning this into a macro looks like:
#lang racket
(require (for-syntax syntax/parse))
(define-syntax (replace-id stx)
(syntax-parse stx
[(_replace-id from to so)
(syntax/loc stx
(with-syntax ([from #'to])
#'so))]))
(replace-id v1 v4 #'(or (and v1 v2) (and v1 v3)))
If you want to use replace-id in a macro, then wrap begin-syntax around
the definition in order to define it in phase 1.
If Racket's match macro were a function I could do this:
(define my-clauses (list '[(list '+ x y) (list '+ y x)]
'[_ 42]))
(on-user-input
(λ (user-input)
(define expr (get-form-from-user-input user-input)) ; expr could be '(+ 1 2), for example.
(apply match expr my-clauses)))
I think there are two very different ways to do this. One is to move my-clauses into macro world, and make a macro something like this (doesn't work):
(define my-clauses (list '[(list '+ x y) (list '+ y x)]
'[_ 42]))
(define-syntax-rule (match-clauses expr)
(match expr my-clauses)) ; this is not the way it's done.
; "Macros that work together" discusses this ideas, right? I'll be reading that today.
(on-user-input
(λ (user-input)
(define expr (get-form-from-user-input user-input)) ; expr could be '(+ 1 2), for example.
(match-clauses expr)))
The alternative, which might be better in the end because it would allow me to change my-clauses at runtime, would be to somehow perform the pattern matching at runtime. Is there any way I can use match on runtime values?
In this question Ryan Culpepper says
It's not possible to create a function where the formal parameters and body are given as run-time values (S-expressions) without using eval.
So I guess I'd have to use eval, but the naive way won't work because match is a macro
(eval `(match ,expr ,#my-clauses) (current-namespace))
I got the desired result with the following voodoo from the guide
(define my-clauses '([(list'+ x y) (list '+ y x)]
[_ 42]))
(define-namespace-anchor a)
(define ns (namespace-anchor->namespace a))
(eval `(match '(+ 1 2) ,#my-clauses) ns) ; '(+ 2 1)
Is the pattern matching happening at runtime now? Is it a bad idea?
To answer the first part of your question (assuming you don't necessarily need the match clauses to be supplied at runtime):
The key is to:
Define my-clauses for compile time ("for syntax").
Reference that correctly in the macro template.
So:
(begin-for-syntax
(define my-clauses (list '[(list '+ x y) (list '+ y x)]
'[_ 42])))
(define-syntax (match-clauses stx)
(syntax-case stx ()
[(_ expr) #`(match expr #,#my-clauses)]))
The pattern matching is happening at runtime in the last example.
One way to check is to look at the expansion:
> (syntax->datum
(expand '(eval `(match '(+ 1 2) ,#my-clauses) ns)))
'(#%app eval (#%app list* 'match ''(+ 1 2) my-clauses) ns)
Whether is a good idea...
Using eval is rather slow, so if you call it often it might be better to find another solution. If you haven't seen it already you might want to read "On eval in dynamic languages generally and in Racket specifically." on the Racket blog.
Thank you both very much, your answers gave me much food for thought. What I am trying to do is still not very well defined, but I seem to be learning a lot in the process, so that's good.
The original idea was to make an equation editor that is a hybrid between paredit and a computer algebra system. You enter an initial math s-expression, e.g. (+ x (* 2 y) (^ (- y x) 2). After that the program presents you with a list of step transformations that you would normally make by hand: substitute a variable, distribute, factor, etc. Like a CAS, but one step at a time. Performing a transformation would happen when the user presses the corresponding key combination, although one possibility is to just show a bunch of possible results, and let the user choose the new state of the expression amongst them. For UI charterm will do for now.
At first I thought I would have the transformations be clauses in a match expression, but now I think I'll make them functions that take and return s-expressions. The trouble with choosing compile time vs runtime is that I want the user to be able to add more transformations, and choose his own keybindings. That could mean that they write some code which I require, or they require mine, before the application is compiled, so it doesn't force me to use eval. But it may be best if I give the user a REPL so he has programmatic control of the expression and his interactions with it as well.
Anyway, today I got caught up reading about macros, evaluation contexts and phases. I'm liking racket more and more and I'm still to investigate about making languages... I will switch to tinkering mode now and see if I get some basic form of what I'm describing to work before my head explodes with new ideas.
Suppose I want to trigger a Scheme macro on something other than the first item in an s-expression. For example, suppose that I wanted to replace define with an infix-style :=, so that:
(a := 5) -> (define a 5)
((square x) := (* x x)) -> (define (square x) (* x x))
The actual transformation seems to be quite straightforward. The trick will be getting Scheme to find the := expressions and macro-expand them. I've thought about surrounding large sections of code that use the infix syntax with a standard macro, maybe: (with-infix-define expr1 expr2 ...), and having the standard macro walk through the expressions in its body and perform any necessary transformations. I know that if I take this approach, I'll have to be careful to avoid transforming lists that are actually supposed to be data, such as quoted lists, and certain sections of quasiquoted lists. An example of what I envision:
(with-infix-define
((make-adder n) := (lambda (m) (+ n m)))
((foo) :=
(add-3 := (make-adder 3))
(add-6 := (make-adder 6))
(let ((a 5) (b 6))
(+ (add-3 a) (add-6 b))))
(display (foo))
(display '(This := should not be transformed))
So, my question is two-fold:
If I take the with-infix-define route, do I have to watch out for any stumbling blocks other than quote and quasiquote?
I feel a bit like I'm reinventing the wheel. This type of code walk seems like exactly what standard macro expanding systems would have to do - the only difference is that they only look at the first item in a list when deciding whether or not to do any code transformation. Is there any way I can just piggyback on existing systems?
Before you continue with this, it's best to think things over thoroughly -- IME you'd often find that what you really want a reader-level handling of := as an infix syntax. That will of course mean that it's also infix in quotations etc, so it would seem bad for now, but again, my experience is that you end up realizing that it's better to do things consistently.
For completeness, I'll mention that in Racket there's a read-syntax hack for infix-like expressions: (x . define . 1) is read as (define x 1). (And as above, it works everywhere.)
Otherwise, your idea of a wrapping macro is pretty much the only thing you can do. This doesn't make it completely hopeless though, you might have a hook into your implementation's expander that can allow you to do such things -- for example, Racket has a special macro called #%module-begin that wraps a complete module body and #%top-interaction that wraps toplevel expressions on the REPL. (Both of these are added implicitly in the two contexts.) Here's an example (I'm using Racket's define-syntax-rule for simplicity):
#lang racket/base
(provide (except-out (all-from-out racket/base)
#%module-begin #%top-interaction)
(rename-out [my-module-begin #%module-begin]
[my-top-interaction #%top-interaction]))
(define-syntax infix-def
(syntax-rules (:= begin)
[(_ (begin E ...)) (begin (infix-def E) ...)]
[(_ (x := E ...)) (define x (infix-def E) ...)]
[(_ E) E]))
(define-syntax-rule (my-module-begin E ...)
(#%module-begin (infix-def E) ...))
(define-syntax-rule (my-top-interaction . E)
(#%top-interaction . (infix-def E)))
If I put this in a file called my-lang.rkt, I can now use it as follows:
#lang s-exp "my-lang.rkt"
(x := 10)
((fib n) :=
(done? := (<= n 1))
(if done? n (+ (fib (- n 1)) (fib (- n 2)))))
(fib x)
Yes, you need to deal with a bunch of things. Two examples in the above are handling begin expressions and handling function bodies. This is obviously a very partial list -- you'll also want bodies of lambda, let, etc. But this is still better than some blind massaging, since that's just not practical as you can't really tell in advance how some random piece of code will end up. As an easy example, consider this simple macro:
(define-syntax-rule (track E)
(begin (eprintf "Evaluating: ~s\n" 'E)
E))
(x := 1)
The upshot of this is that for a proper solution, you need some way to pre-expand the code, so that you can then scan it and deal with the few known core forms in your implmenetation.
Yes, all of this is repeating work that macro expanders do, but since you're changing how expansion works, there's no way around this. (To see why it's a fundamental change, consider something like (if := 1) -- is this a conditional expression or a definition? How do you decide which one takes precedence?) For this reason, for languages with such "cute syntax", a more popular approach is to read and parse the code into plain S-expressions, and then let the actual language implementation use plain functions and macros.
Redefining define is a little complicated. See #Eli's excellent explanation.
If on the other hand, you are content with := to use set! things are a little simpler.
Here is a small example:
#lang racket
(module assignment racket
(provide (rename-out [app #%app]))
(define-syntax (app stx)
(syntax-case stx (:=)
[(_ id := expr)
(identifier? #'id)
(syntax/loc stx (set! id expr))]
[(_ . more)
(syntax/loc stx (#%app . more))])))
(require 'assignment)
(define x 41)
(x := (+ x 1))
(displayln x)
To keep the example to a single file, I used submodules (available in the prerelease version of Racket).