I'm trying to write a macro that builds middleware akin to that used in compojure.
I want to be able to call:
(def-middleware plus-two [x]
(+ 2 x))
And have the result look like:
(defn plus-two [f]
(fn [x]
(f (+ 2 x)))
I've got this far from reading on-line guides but it's not working out for me:
(defmacro def-middleware [fn-name args & body]
'(defn ~fn-name [f]
(fn ~args
(f ~#body))))
Any help or a pointer to a better macro writing guide would be great, thanks.
Let's see what macroexpand-1 gives us:
user=> (clojure.pprint/pprint (macroexpand-1 '(def-middleware plus-two [x] (+ 2 x))))
(defn
~fn-name
[f]
$
(fn ~args$ (f (clojure.core/unquote-splicing body))))
Not precisely what we're after! First things first: if you want to unquote things in a macro, you need to use "`" (the quasiquote/syntax-quote operator), not "'". Also, I'm not sure what you're after with those dollar signs, but it might be that you're going for the clojure's handy shortcut for using gensym for hygiene. But you need it immediately after each identifier you use it with (so [f#], not [f]$), and you need it on each occurrence of the identifier. And you don't need it with args. Putting all this together:
user=> (defmacro def-middleware [fn-name args & body] `(defn ~fn-name [f#] (fn ~args (f# ~#body))))
#'user/def-middleware
user=> (clojure.pprint/pprint (macroexpand-1 '(def-middleware plus-two [x] (+ 2 x))))
(clojure.core/defn
plus-two
[f__594__auto__]
(clojure.core/fn [x] (f__594__auto__ (+ 2 x))))
nil
user=>
Perhaps this is just a macro exercise, but the specific example of defining a middleware function is not good - you lose a lot of the flexibility that the middleware concept gives you. For example, why should this evaluate (f (+ 2 x)) instead of (+ 2 (f x))? Both are sensible ways to wrap this function, and your syntax gives no way to describe the latter. Really just defining a function-returning-function is flexible, simple, and easy; this macro brings little to the table.
Related
In one variant of Common Lisp (I think it was CMUCL, but I might be wrong—I can't find it any more) there was a function that was (I think) called function-lambda-expression. If it got a procedure, it would print out the lambda expression that had generated it. Example:
(let ((my-thunk (lambda () (+ 1 2))))
(write my-thunk)
(write (function-lambda-expression my-thunk)))
This would print out something like:
#<PROCEDURE>
(LAMBDA () (+ 1 2))
It was terribly useful for debugging and exploring the language.
I'm looking for a function like this in Racket. I've looked through the Racket Documentation but I can't find anything like it. (I wouldn't be surprised if I overlooked it, however.) Is there an equivalent in Racket?
No. Racket's lambda produces a closure that does not remember its S-expression (or syntax object) form. It does typically remember its name (or its abbreviated source location, if no name can be inferred), and that's often enough to help with debugging. (See object-name.)
You can build your own variant of lambda that has this feature, using Racket's applicable structures and a simple macro. Here's a basic example:
#lang racket
(struct exp-closure (f exp)
#:property prop:procedure (struct-field-index f))
(define-syntax-rule (exp-lambda formals . body)
(exp-closure (lambda formals . body)
(quote (exp-lambda formals . body))))
(let ([my-thunk (exp-lambda () (+ 1 2))])
(printf "fun is ~v\n" my-thunk)
(printf "exp is ~v\n" (exp-closure-exp my-thunk))
(printf "result is ~v\n" (my-thunk)))
This produces
fun is #<procedure:...tmp/lambda.rkt:11:19>
exp is '(exp-lambda () (+ 1 2))
result is 3
A better version of this macro might propagate the source location of the macro use to the lambda expression it creates, or the inferred name (see syntax-local-infer-name), or both.
I'm in a process of implementing Hygienic macros in my Scheme implementation, I've just implemented syntax-rules, but I have this code:
(define odd?
(syntax-rules ()
((_ x) (not (even? x)))))
what should be the difference between that and this:
(define-syntax odd?
(syntax-rules ()
((_ x) (not (even? x)))))
from what I understand syntax-rules just return syntax transformer, why you can't just use define to assign that to symbol? Why I need to use define-syntax? What extra stuff that expression do?
Should first also work in scheme? Or only the second one?
Also what is the difference between let vs let-syntax and letrec vs letrec-syntax. Should (define|let|letrec)-syntax just typecheck if the value is syntax transformer?
EDIT:
I have this implementation, still using lisp macros:
;; -----------------------------------------------------------------------------
(define-macro (let-syntax vars . body)
`(let ,vars
,#(map (lambda (rule)
`(typecheck "let-syntax" ,(car rule) "syntax"))
vars)
,#body))
;; -----------------------------------------------------------------------------
(define-macro (letrec-syntax vars . body)
`(letrec ,vars
,#(map (lambda (rule)
`(typecheck "letrec-syntax" ,(car rule) "syntax"))
vars)
,#body))
;; -----------------------------------------------------------------------------
(define-macro (define-syntax name expr)
(let ((expr-name (gensym)))
`(define ,name
(let ((,expr-name ,expr))
(typecheck "define-syntax" ,expr-name "syntax")
,expr-name))))
This this code correct?
Should this code works?
(let ((let (lambda (x) x)))
(let-syntax ((odd? (syntax-rules ()
((_ x) (not (even? x))))))
(odd? 11)))
This question seems to imply some deep confusion about macros.
Let's imagine a language where syntax-rules returns some syntax transformer function (I am not sure this has to be true in RnRS Scheme, it is true in Racket I think), and where let and let-syntax were the same.
So let's write this function:
(define (f v)
(let ([g v])
(g e (i 10)
(if (= i 0)
i
(e (- i 1))))))
Which we can turn into this, of course:
(define (f v n)
(v e (i n)
(if (<= i 0)
i
(e (- i 1)))))
And I will tell you in addition that there is no binding for e or i in the environment.
What is the interpreter meant to do with this definition? Could it compile it? Could it safely infer that i can't possibly make any sense since it is used as a function and then as a number? Can it safely do anything at all?
The answer is that no, it can't. Until it knows what the argument to the function is it can't do anything. And this means that each time f is called it has to make that decision again. In particular, v might be:
(syntax-rules ()
[(_ name (var init) form ...)
(letrec ([name (λ (var)
form ...)])
(name init))]))
Under which the definition of f does make some kind of sense.
And things get worse: much worse. How about this?
(define (f v1 v2 n)
(let ([v v1])
(v e (i n)
...
(set! v (if (eq? v v1) v2 v1))
...)))
What this means is that a system like this wouldn't know what the code it was meant to interpret meant until, the moment it was interpreting it, or even after that point, as you can see from the second function above.
So instead of this horror, Lisps do something sane: they divide the process of evaluating bits of code into phases where each phase happens, conceptually, before the next one.
Here's a sequence for some imagined Lisp (this is kind of close to what CL does, since most of my knowledge is of that, but it is not intended to represent any particular system):
there's a phase where the code is turned from some sequence of characters to some object, possibly with the assistance of user-defined code;
there's a phase where that object is rewritten into some other object by user- and system-defined code (macros) – the result of this phase is something which is expressed in terms of functions and some small number of primitive special things, traditionally called 'special forms' which are known to the processes of stage 3 and 4;
there may be a phase where the object from phase 2 is compiled, and that phase may involve another set of user-defined macros (compiler macros);
there is a phase where the resulting code is evaluated.
And for each unit of code these phases happen in order, each phase completes before the next one begins.
This means that each phase in which the user can intervene needs its own set of defining and binding forms: it needs to be possible to say that 'this thing controls what happens at phase 2' for instance.
That's what define-syntax, let-syntax &c do: they say that 'these bindings and definitions control what happens at phase 2'. You can't, for instance, use define or let to do that, because at phase 2, these operations don't yet have meaning: they gain meaning (possibly by themselves being macros which expand to some primitive thing) only at phase 3. At phase 2 they are just bits of syntax which the macro is ingesting and spitting out.
I've got another question involving self-reference in Common Lisp. I found a thread on Stack Exchange which poses a problem of writing the shortest program that would print all printable ASCII characters NOT present in the program's source code. This got me thinking how to tackle the problem in Common Lisp. I hit against two problems - one probably trivial, the other more tricky:
First is the case of writing a CL script, e.g. starting with #!/usr/bin/env sbcl --script. I thought that through *posix-argv* I could access all command line arguments including the name of the called script. I also looked for the equivalent of Bash $0 but could find none. What worked for me in the end is this ugly Bash-ified SBCL script, which explicitly passes $0 to SBCL and proceeds from that:
#!/bin/bash
#|
sbcl --script $0 $0
exit
|#
(defun file-string (path)
(with-open-file (stream path)
(let ((data (make-string (file-length stream))))
(read-sequence data stream)
data)))
(let* ((printable (mapcar #'code-char (loop for i from #x20 to #x7e collect i)))
(absent (set-difference
printable
(coerce (file-string (cadr *posix-argv*)) 'list))))
(print (coerce absent 'string)))
My question regarding this point is: can you think of any way of doing it without relying so heavily on Bash supplying relevant arguments? Or, more briefly: is there a CL (SBCL in particular) equivalent of $0?
Now comes the part that I'm totally puzzled with. Before resorting to the script approach above I tried to accomplish this goal in a more REPL-oriented way. Based on the &whole specifier in defmacro and considerations in this thread I've tried to get the name of the macro from the &whole argument and somehow "read in" its source. And I have absolutely no idea how to do it. So in short: given the name of the macro, can I somehow obtain the defmacro form which defined it? And I'm talking about a generic solution, rather than parsing the REPL history.
EDIT: Regarding mbratch's question about use of macroexpand-1 here's how I do it:
(defmacro self-refer (&whole body)
(macroexpand-1 `',body))
With this call I'm able to obtain (SELF-REFER) by calling (SELF-REFER). Which isn't much of a solution...
I hope someone could point me in the right direction. Thanks!
Getting the source of a macro is not defined in Common Lisp.
This may work (Example from LispWorks):
CL-USER 10 > (defmacro foo (a b) `(* (+ ,a ,b) (+ ,a ,a)))
FOO
CL-USER 11 > (pprint (function-lambda-expression (macro-function 'foo)))
(LAMBDA
(DSPEC::%%MACROARG%% #:&ENVIRONMENT1106 &AUX (#:&WHOLE1107 DSPEC::%%MACROARG%%)
(#:\(A\ ...\)1108 (CDR #:&WHOLE1107))
(#:CHECK-LAMBDA-LIST-TOP-LEVEL1110
(DSPEC::CHECK-LAMBDA-LIST-TOP-LEVEL '(A B)
#:&WHOLE1107
#:\(A\ ...\)1108
2
2
'NIL
:MACRO))
(A (CAR (DSPEC::THE-CONS #:\(A\ ...\)1108)))
(#:\(B\)1109 (CDR (DSPEC::THE-CONS #:\(A\ ...\)1108)))
(B (CAR (DSPEC::THE-CONS #:\(B\)1109))))
(DECLARE (LAMBDA-LIST A B))
(BLOCK FOO `(* (+ ,A ,B) (+ ,A ,A))))
An even more esoteric way is to alter the existing DEFMACRO to record its source.
Many Lisp implementations have a non-standard feature called advising. LispWorks for example can advise macros:
CL-USER 31 > (defadvice (defmacro source-record-defmacro :after)
(&rest args)
(setf (get (second (first args)) :macro-source) (first args)))
T
Above adds code to the standard DEFMACRO macro, which records the source on the symbol property list of the macro name. defmacro is the name of the thing to advise. source-record-defmacro is the chosen name of this advice. :after then specifies that the code should run after the normal defmacro code.
CL-USER 32 > (defmacro foo (a b) `(* (+ ,a ,b) (+ ,a ,a)))
FOO
CL-USER 33 > (pprint (get 'foo :macro-source))
(DEFMACRO FOO (A B) `(* (+ ,A ,B) (+ ,A ,A)))
Again, this is completely non-standard - I'm not sure if a comparable mechanism exists for SBCL, though it has something called 'encapsulate'.
A very belated followup to Rainer Joswig's LispWorks solution. I've been using Allegro CL lately and discovered the fwrap facility. Conceptually it's very similar to the defadvice above and slighly more verbose. Here's a re-iteration of Rainer's example in ACL 10.0:
(def-fwrapper source-record-defmacro (&rest args)
(setf (get (second (first args)) :macro-source) (first args))
(call-next-fwrapper))
Having defined an fwrapper you need to "put it into action" explicitly:
(fwrap 'defmacro 'srd 'source-record-defmacro)
After this it's like in Rainer's example:
CL-USER> (defmacro foo (a b) `(* (+ ,a ,b) (+ ,a ,a)))
FOO
CL-USER> (pprint (get 'foo :macro-source))
(DEFMACRO FOO (A B) `(* (+ ,A ,B) (+ ,A ,A)))
; No value
I am trying to solve the last part of question 4.4 of the Structure and Interpretation of computer programming; the task is to implement or as a syntactic transformation. Only elementary syntactic forms are defined; quote, if, begin, cond, define, apply and lambda.
(or a b ... c) is equal to the first true value or false if no value is true.
The way I want to approach it is to transform for example (or a b c) into
(if a a (if b b (if c c false)))
the problem with this is that a, b, and c would be evaluated twice, which could give incorrect results if any of them had side-effects. So I want something like a let
(let ((syma a))
(if syma syma (let ((symb b))
(if symb symb (let ((symc c))
(if (symc symc false)) )) )) )
and this in turn could be implemented via lambda as in Exercise 4.6. The problem now is determining symbols syma, symb and symc; if for example the expression b contains a reference to the variable syma, then the let will destroy the binding. Thus we must have that syma is a symbol not in b or c.
Now we hit a snag; the only way I can see out of this hole is to have symbols that cannot have been in any expression passed to eval. (This includes symbols that might have been passed in by other syntactic transformations).
However because I don't have direct access to the environment at the expression I'm not sure if there is any reasonable way of producing such symbols; I think Common Lisp has the function gensym for this purpose (which would mean sticking state in the metacircular interpreter, endangering any concurrent use).
Am I missing something? Is there a way to implement or without using gensym? I know that Scheme has it's own hygenic macro system, but I haven't grokked how it works and I'm not sure whether it's got a gensym underneath.
I think what you might want to do here is to transform to a syntactic expansion where the evaluation of the various forms aren't nested. You could do this, e.g., by wrapping each form as a lambda function and then the approach that you're using is fine. E.g., you can do turn something like
(or a b c)
into
(let ((l1 (lambda () a))
(l2 (lambda () b))
(l3 (lambda () c)))
(let ((v1 (l1)))
(if v1 v1
(let ((v2 (l2)))
(if v2 v2
(let ((v3 (l3)))
(if v3 v3
false)))))))
(Actually, the evaluation of the lambda function calls are still nested in the ifs and lets, but the definition of the lambda functions are in a location such that calling them in the nested ifs and lets doesn't cause any difficulty with captured bindings.) This doesn't address the issue of how you get the variables l1–l3 and v1–v3, but that doesn't matter so much, none of them are in scope for the bodies of the lambda functions, so you don't need to worry about whether they appear in the body or not. In fact, you can use the same variable for all the results:
(let ((l1 (lambda () a))
(l2 (lambda () b))
(l3 (lambda () c)))
(let ((v (l1)))
(if v v
(let ((v (l2)))
(if v v
(let ((v (l3)))
(if v v
false)))))))
At this point, you're really just doing loop unrolling of a more general form like:
(define (functional-or . functions)
(if (null? functions)
false
(let ((v ((first functions))))
(if v v
(functional-or (rest functions))))))
and the expansion of (or a b c) is simply
(functional-or (lambda () a) (lambda () b) (lambda () c))
This approach is also used in an answer to Why (apply and '(1 2 3)) doesn't work while (and 1 2 3) works in R5RS?. And none of this required any GENSYMing!
In SICP you are given two ways of implementing or. One that handles them as special forms which is trivial and one as derived expressions. I'm unsure if they actually thought you would see this as a problem, but you can do it by implementing gensym or altering variable? and how you make derived variables like this:
;; a unique tag to identify special variables
(define id (vector 'id))
;; a way to make such variable
(define (make-var x)
(list id x))
;; redefine variable? to handle macro-variables
(define (variable? exp)
(or (symbol? exp)
(tagged-list? exp id)))
;; makes combinations so that you don't evaluate
;; every part twice in case of side effects (set!)
(define (or->combination terms)
(if (null? terms)
'false
(let ((tmp (make-var 'tmp)))
(list (make-lambda (list tmp)
(list (make-if tmp
tmp
(or->combination (cdr terms)))))
(car terms)))))
;; My original version
;; This might not be good since it uses backquotes not introduced
;; until chapter 5 and uses features from exercise 4.6
;; Though, might be easier to read for some so I'll leave it.
(define (or->combination terms)
(if (null? terms)
'false
(let ((tmp (make-var 'tmp)))
`(let ((,tmp ,(car terms)))
(if ,tmp
,tmp
,(or->combination (cdr terms)))))))
How it works is that make-var creates a new list every time it is called, even with the same argument. Since it has id as it's first element variable? will identify it as a variable. Since it's a list it will only match in variable lookup with eq? if it is the same list, so several nested or->combination tmp-vars will all be seen as different by lookup-variable-value since (eq? (list) (list)) => #f and special variables being lists they will never shadow any symbol in code.
This is influenced by eiod, by Al Petrofsky, which implements syntax-rules in a similar manner. Unless you look at others implementations as spoilers you should give it a read.
Suppose I want to trigger a Scheme macro on something other than the first item in an s-expression. For example, suppose that I wanted to replace define with an infix-style :=, so that:
(a := 5) -> (define a 5)
((square x) := (* x x)) -> (define (square x) (* x x))
The actual transformation seems to be quite straightforward. The trick will be getting Scheme to find the := expressions and macro-expand them. I've thought about surrounding large sections of code that use the infix syntax with a standard macro, maybe: (with-infix-define expr1 expr2 ...), and having the standard macro walk through the expressions in its body and perform any necessary transformations. I know that if I take this approach, I'll have to be careful to avoid transforming lists that are actually supposed to be data, such as quoted lists, and certain sections of quasiquoted lists. An example of what I envision:
(with-infix-define
((make-adder n) := (lambda (m) (+ n m)))
((foo) :=
(add-3 := (make-adder 3))
(add-6 := (make-adder 6))
(let ((a 5) (b 6))
(+ (add-3 a) (add-6 b))))
(display (foo))
(display '(This := should not be transformed))
So, my question is two-fold:
If I take the with-infix-define route, do I have to watch out for any stumbling blocks other than quote and quasiquote?
I feel a bit like I'm reinventing the wheel. This type of code walk seems like exactly what standard macro expanding systems would have to do - the only difference is that they only look at the first item in a list when deciding whether or not to do any code transformation. Is there any way I can just piggyback on existing systems?
Before you continue with this, it's best to think things over thoroughly -- IME you'd often find that what you really want a reader-level handling of := as an infix syntax. That will of course mean that it's also infix in quotations etc, so it would seem bad for now, but again, my experience is that you end up realizing that it's better to do things consistently.
For completeness, I'll mention that in Racket there's a read-syntax hack for infix-like expressions: (x . define . 1) is read as (define x 1). (And as above, it works everywhere.)
Otherwise, your idea of a wrapping macro is pretty much the only thing you can do. This doesn't make it completely hopeless though, you might have a hook into your implementation's expander that can allow you to do such things -- for example, Racket has a special macro called #%module-begin that wraps a complete module body and #%top-interaction that wraps toplevel expressions on the REPL. (Both of these are added implicitly in the two contexts.) Here's an example (I'm using Racket's define-syntax-rule for simplicity):
#lang racket/base
(provide (except-out (all-from-out racket/base)
#%module-begin #%top-interaction)
(rename-out [my-module-begin #%module-begin]
[my-top-interaction #%top-interaction]))
(define-syntax infix-def
(syntax-rules (:= begin)
[(_ (begin E ...)) (begin (infix-def E) ...)]
[(_ (x := E ...)) (define x (infix-def E) ...)]
[(_ E) E]))
(define-syntax-rule (my-module-begin E ...)
(#%module-begin (infix-def E) ...))
(define-syntax-rule (my-top-interaction . E)
(#%top-interaction . (infix-def E)))
If I put this in a file called my-lang.rkt, I can now use it as follows:
#lang s-exp "my-lang.rkt"
(x := 10)
((fib n) :=
(done? := (<= n 1))
(if done? n (+ (fib (- n 1)) (fib (- n 2)))))
(fib x)
Yes, you need to deal with a bunch of things. Two examples in the above are handling begin expressions and handling function bodies. This is obviously a very partial list -- you'll also want bodies of lambda, let, etc. But this is still better than some blind massaging, since that's just not practical as you can't really tell in advance how some random piece of code will end up. As an easy example, consider this simple macro:
(define-syntax-rule (track E)
(begin (eprintf "Evaluating: ~s\n" 'E)
E))
(x := 1)
The upshot of this is that for a proper solution, you need some way to pre-expand the code, so that you can then scan it and deal with the few known core forms in your implmenetation.
Yes, all of this is repeating work that macro expanders do, but since you're changing how expansion works, there's no way around this. (To see why it's a fundamental change, consider something like (if := 1) -- is this a conditional expression or a definition? How do you decide which one takes precedence?) For this reason, for languages with such "cute syntax", a more popular approach is to read and parse the code into plain S-expressions, and then let the actual language implementation use plain functions and macros.
Redefining define is a little complicated. See #Eli's excellent explanation.
If on the other hand, you are content with := to use set! things are a little simpler.
Here is a small example:
#lang racket
(module assignment racket
(provide (rename-out [app #%app]))
(define-syntax (app stx)
(syntax-case stx (:=)
[(_ id := expr)
(identifier? #'id)
(syntax/loc stx (set! id expr))]
[(_ . more)
(syntax/loc stx (#%app . more))])))
(require 'assignment)
(define x 41)
(x := (+ x 1))
(displayln x)
To keep the example to a single file, I used submodules (available in the prerelease version of Racket).