Imagine the following code to dynamically create a macro:
(def a (list '+ 1 2))
(def b (list '- 10 5))
(def c (list '/ 22 2))
(defmacro gg [h]
(let [k# `~h]
k#))
The intent is to pass a vector of symbols to a macro, do some evaluation on each element of the vector such that it returns a nice macro-esque form, then have the macro combine them into a nice macro and evaluate it. The above example works all except the actual evaluation.
When I run it I get:
(gg [a b c])
=> [(+ 1 2) (- 10 5) (/ 22 2)]
What is the secret to passing a symbol that is a list of symbols and getting a macro to evaluate them? I have tried lots of combinations of quoting and have yet to hit the right one.
The real purpose of this question is to build an Archimedes Ogre query based on a definition of a path through the graph. If someone has an example of that, I would be grateful.
EDIT:
(defmacro gg2 [h]
`(do ~#(map identity h)))
(macroexpand '(gg2 [a b c]))
=> (do a b c)
(gg2 [a b c])
=> (/ 22 2)
I was hoping to get 11 rather than the form.
You don't need a macro. Macros don't do what you're looking for here. What you are looking for is eval.
(def a '/)
(def b 22)
(def c 2)
(eval (list* [a b c]))
=> 11
Of course, you can write a macro which expands into (eval (list* ...)) if you want. It could just as well be a function though.
This is a very common mistake when starting out with macros; trying to write a macro which depends on the run-time value of its arguments. Macros run at compile-time, and generally the values of the symbols which you pass to a macro are not yet available when the macro is expanded.
About the use of eval, some cautions are in order. No one said it better than Paul Graham:
Generally it is not a good idea to call eval at runtime, for two reasons:
It’s inefficient: eval is handed a raw list, and either has to compile it on the spot, or evaluate it in an interpreter. Either way is slower than compiling the code beforehand, and just calling it.
It’s less powerful, because the expression is evaluated with no lexical context. Among other things, this means that you can’t refer to ordinary variables visible outside the expression being evaluated.
Usually, calling eval explicitly is like buying something in an airport gift-shop. Having waited till the last moment, you have to pay high prices for a limited selection of second-rate goods.
Related
AND and OR are macros and since macros aren't first class in scheme/racket they cannot be passed as arguments to other functions. A partial solution is to use and-map or or-map. Is it possible to write a function that would take arbitrary macro and turn it into a function so that it can be passed as an argument to another function? Are there any languages that have first class macros?
In general, no. Consider that let is (or could be) implemented as a macro on top of lambda:
(let ((x 1))
(foo x))
could be a macro that expands to
((lambda (x) (foo x)) 1)
Now, what would it look like to convert let to a function? Clearly it is nonsense. What would its inputs be? Its return value?
Many macros will be like this. In fact, any macro that could be routinely turned into a function without losing any functionality is a bad macro! Such a macro should have been a function to begin with.
I agree with #amalloy. If something is written as a macro, it probably does something that functions can't do (e.g., introduce bindings, change evaluation order). So automatically converting arbitrary macro into a function is a really bad idea even if it is possible.
Is it possible to write a function that would take arbitrary macro and turn it into a function so that it can be passed as an argument to another function?
No, but it is somewhat doable to write a macro that would take some macro and turn it into a function.
#lang racket
(require (for-syntax racket/list))
(define-syntax (->proc stx)
(syntax-case stx ()
[(_ mac #:arity arity)
(with-syntax ([(args ...) (generate-temporaries (range (syntax-e #'arity)))])
#'(λ (args ...) (mac args ...)))]))
((->proc and #:arity 2) 42 12)
(apply (->proc and #:arity 2) '(#f 12))
((->proc and #:arity 2) #f (error 'not-short-circuit))
You might also be interested in identifier macro, which allows us to use an identifier as a macro in some context and function in another context. This could be used to create a first class and/or which short-circuits when it's used as a macro, but could be passed as a function value in non-transformer position.
On the topic of first class macro, take a look at https://en.wikipedia.org/wiki/Fexpr. It's known to be a bad idea.
Not in the way you probably expect
To see why, here is a way of thinking about macros: A macro is a function which takes a bit of source code and turns it into another bit of source code: the expansion of the macro. In other words a macro is a function whose domain and range are source code.
Once the source code is fully expanded, then it's fed to either an evaluator or a compiler. Let's assume it's fed to a compiler because it makes the question easier to answer: a compiler itself is simply a function whose domain is source code and whose range is some sequence of instructions for a machine (which may or may not be a real machine) to execute. Those instructions might include things like 'call this function on these arguments'.
So, what you are asking is: can the 'this function' in 'call this function on these arguments' be some kind of macro? Well, yes, it could be, but whatever source code it is going to transform certainly can not be the source code of the program you are executing, because that is gone: all that's left is the sequence of instructions that was the return value of the compiler.
So you might say: OK, let's say we disallow compilers: can we do it now? Well, leaving aside that 'disallowing compilers' is kind of a serious limitation, this was, in fact, something that very old dialects of Lisp sort-of did, using a construct called a FEXPR, as mentioned in another answer. It's important to realise that FEXPRs existed because people had not yet invented macros. Pretty soon, people did invent macros, and although FEXPRs and macros coexisted for a while – mostly because people had written code which used FEXPRs which they wanted to keep running, and because writing macros was a serious pain before things like backquote existed – FEXPRs died out. And they died out because they were semantically horrible: even by the standards of 1960s Lisps they were semantically horrible.
Here's one small example of why FEXPRs are so horrible: Let's say I write this function in a language with FEXPRs:
(define (foo f g x)
(apply f (g x)))
Now: what happens when I call foo? In particular, what happens if f might be a FEXPR?. Well, the answer is that I can't compile foo at all: I have to wait until run-time and make some on-the-fly decision about what to do.
Of course this isn't what these old Lisps with FEXPRs probably did: they would just silently have assumed that f was a normal function (which they would have called an EXPR) and compiled accordingly (and yes, even very old Lisps had compilers). If you passed something which was a FEXPR you just lost: either the thing detected that, or more likely it fall over horribly or gave you some junk answer.
And this kind of horribleness is why macros were invented: macros provide a semantically sane approach to processing Lisp code which allows (eventually, this took a long time to actually happen) minor details like compilation being possible at all, code having reasonable semantics and compiled code having the same semantics as interpreted code. These are features people like in their languages, it turns out.
Incidentally, in both Racket and Common Lisp, macros are explicitly functions. In Racket they are functions which operate on special 'syntax' objects because that's how you get hygiene, but in Common Lisp, which is much less hygienic, they're just functions which operate on CL source code, where the source code is simply made up of lists, symbols &c.
Here's an example of this in Racket:
> (define foo (syntax-rules ()
[(_ x) x]))
> foo
#<procedure:foo>
OK, foo is now just an ordinary function. But it's a function whose domain & range are Racket source code: it expects a syntax object as an argument and returns another one:
> (foo 1)
; ?: bad syntax
; in: 1
; [,bt for context]
This is because 1 is not a syntax object.
> (foo #'(x 1))
#<syntax:readline-input:5:10 1>
> (syntax-e (foo #'(x 1)))
1
And in CL this is even easier to see: Here's a macro definition:
(defmacro foo (form) form)
And now I can get hold of the macro's function and call it on some CL source code:
> (macro-function 'foo)
#<Function foo 4060000B6C>
> (funcall (macro-function 'foo) '(x 1) nil)
1
In both Racket and CL, macros are, in fact, first-class (or, in the case of Racket: almost first-class, I think): they are functions which operate on source code, which itself is first-class: you can write Racket and CL programs which construct and manipulate source code in arbitrary ways: that's what macros are in these languages.
In the case of Racket I have said 'almost first-class', because I can't see a way, in Racket, to retrieve the function which sits behind a macro defined with define-syntax &c.
I've created something like this in Scheme, it's macro that return lambda that use eval to execute the macro:
(define-macro (macron m)
(let ((x (gensym)))
`(lambda (,x)
(eval `(,',m ,#,x)))))
Example usage:
;; normal eval
(define x (map (lambda (x)
(eval `(lambda ,#x)))
'(((x) (display x)) ((y) (+ y y)))))
;; using macron macro
(define x (map (macron lambda)
'(((x) (display x)) ((y) (+ y y)))))
and x in both cases is list of two functions.
another example:
(define-macro (+++ . args)
`(+ ,#args))
((macron +++) '(1 2 3))
Here is simplified example from book On Lisp by Paul Graham (scheme like syntax).
(define-macro (bar)
(let ((x 10) (y '(1 2 3)) (z 'foo))
`(list ,x `(,',z ,,#y))))
I know how ,,#y should work but not sure exactly how ,',z should work what should be evaluated first and in what order. (I know it should evaluate to symbol foo because it return (10 (foo 1 2 3)) in guile, but I'm not sure what are exact steps).
I need this for my lisp in JavaScript where I have result:
(10 ((unquote z) 1 2 3))
because it just evaluate it form left to right (I'm only handling specially ,, and more commas). How should you evaluate this expression.
There is also this example in the book:
(defmacro propmacro (propname)
`(defmacro ,propname (obj)
`(get ,obj ',',propname)))
how ',', should be evaluated? What are the steps in this case?
Are there any other weird edge cases with backquote/quasiquote? Can you show examples of those and how they should be evaluated and in what order?
How ,',z works is that:
`(list ,x `(,',z ,,#y))))
^ ^
| `- this comma
`- belongs to this backquote
The above comma interpolates, into the inner backquote, the expression ',z or (quote ,z). And that ,z, in turn, belongs to the outer backquote.
Thus the value of z is inserted into (quote ,z) to make (quote <value-of-z>).
Then, effectively, the inner backquote then behaves like `(,'<value-of-z>).
Concretely, suppose z contains the list (+ 2 2). Then we can understand it in terms of the outer backquote inserting (+ 2 2) into the inner one to produce `(,'(+ 2 2) ...). This is now straightforward to understand: when the inner backquote is evaluated, the (+ 2 2) is protected from evaluation, resulting in the object ((+ 2 2) ...).
The pattern ,',',', ... ,',expr is used to obtain a single evaluation of expr during the evaluation of the outermost backquote, such that this value is then propagated through any number of evaluation rounds of the remaining backquote nestings without undergoing further evaluation. There is a kind of "backquote algebra" at play here in which the "commas and quotes cancel out".
You can also visualize the ,',','... as a kind of drill bit that digs through the layers of nesting to allow you plant a literal value anywhere in the structure. E.g.
(defmacro super-nested-macro (arg)
`(... `(.... `(.....`(we simply want arg down here ,',',',arg)))))
The author of super-nested-macro just wants to stick the value of arg into the template, in a position that is buried in three other backquotes. Thus the usual ,arg cannot be used: that comma would be misinterpreted as belonging to the inner-most backquote.
Are there any other weird edge cases with backquote/quasiquote?
One weird edge case in backquote is trying to splice into a dot position:
`(a b c . ,#foo) ;; not allowed
`(a b c . ,foo) ;; OK: equivalent to `(a b c ,#foo)
Not sure how various implementations deal with a backquote in a dot position:
`(a b c . `(d e f))
It doesn't really make sense, and I suspect that the actual result obtained will depend on the backquote implementation internals.
Not all objects are traversed for unquoting:
`#c(,(sin theta) ,(cos theta)) ;; Not required by ANSI CL, oops!
This could work via an implementation's extension.
If Racket's match macro were a function I could do this:
(define my-clauses (list '[(list '+ x y) (list '+ y x)]
'[_ 42]))
(on-user-input
(λ (user-input)
(define expr (get-form-from-user-input user-input)) ; expr could be '(+ 1 2), for example.
(apply match expr my-clauses)))
I think there are two very different ways to do this. One is to move my-clauses into macro world, and make a macro something like this (doesn't work):
(define my-clauses (list '[(list '+ x y) (list '+ y x)]
'[_ 42]))
(define-syntax-rule (match-clauses expr)
(match expr my-clauses)) ; this is not the way it's done.
; "Macros that work together" discusses this ideas, right? I'll be reading that today.
(on-user-input
(λ (user-input)
(define expr (get-form-from-user-input user-input)) ; expr could be '(+ 1 2), for example.
(match-clauses expr)))
The alternative, which might be better in the end because it would allow me to change my-clauses at runtime, would be to somehow perform the pattern matching at runtime. Is there any way I can use match on runtime values?
In this question Ryan Culpepper says
It's not possible to create a function where the formal parameters and body are given as run-time values (S-expressions) without using eval.
So I guess I'd have to use eval, but the naive way won't work because match is a macro
(eval `(match ,expr ,#my-clauses) (current-namespace))
I got the desired result with the following voodoo from the guide
(define my-clauses '([(list'+ x y) (list '+ y x)]
[_ 42]))
(define-namespace-anchor a)
(define ns (namespace-anchor->namespace a))
(eval `(match '(+ 1 2) ,#my-clauses) ns) ; '(+ 2 1)
Is the pattern matching happening at runtime now? Is it a bad idea?
To answer the first part of your question (assuming you don't necessarily need the match clauses to be supplied at runtime):
The key is to:
Define my-clauses for compile time ("for syntax").
Reference that correctly in the macro template.
So:
(begin-for-syntax
(define my-clauses (list '[(list '+ x y) (list '+ y x)]
'[_ 42])))
(define-syntax (match-clauses stx)
(syntax-case stx ()
[(_ expr) #`(match expr #,#my-clauses)]))
The pattern matching is happening at runtime in the last example.
One way to check is to look at the expansion:
> (syntax->datum
(expand '(eval `(match '(+ 1 2) ,#my-clauses) ns)))
'(#%app eval (#%app list* 'match ''(+ 1 2) my-clauses) ns)
Whether is a good idea...
Using eval is rather slow, so if you call it often it might be better to find another solution. If you haven't seen it already you might want to read "On eval in dynamic languages generally and in Racket specifically." on the Racket blog.
Thank you both very much, your answers gave me much food for thought. What I am trying to do is still not very well defined, but I seem to be learning a lot in the process, so that's good.
The original idea was to make an equation editor that is a hybrid between paredit and a computer algebra system. You enter an initial math s-expression, e.g. (+ x (* 2 y) (^ (- y x) 2). After that the program presents you with a list of step transformations that you would normally make by hand: substitute a variable, distribute, factor, etc. Like a CAS, but one step at a time. Performing a transformation would happen when the user presses the corresponding key combination, although one possibility is to just show a bunch of possible results, and let the user choose the new state of the expression amongst them. For UI charterm will do for now.
At first I thought I would have the transformations be clauses in a match expression, but now I think I'll make them functions that take and return s-expressions. The trouble with choosing compile time vs runtime is that I want the user to be able to add more transformations, and choose his own keybindings. That could mean that they write some code which I require, or they require mine, before the application is compiled, so it doesn't force me to use eval. But it may be best if I give the user a REPL so he has programmatic control of the expression and his interactions with it as well.
Anyway, today I got caught up reading about macros, evaluation contexts and phases. I'm liking racket more and more and I'm still to investigate about making languages... I will switch to tinkering mode now and see if I get some basic form of what I'm describing to work before my head explodes with new ideas.
I have a simple macro:
(defmacro macrotest [coll]
`(let [result# ~(reduce + coll)]
result#))
Why, if this code works:
(macrotest [1 2 3])
doesn't this code work?
(def mycoll [1 2 3])
(macrotest mycoll)
Symbols are not the same as the value they point to.
One of the important considerations about macros is not just how they create new code, but also how they handle the arguments you pass in.
Consider this:
(def v [1 2 3])
(somefunction v)
The argument v passed to this function does not arrive inside the function as a symbol, v. Instead, because this is a function call, the arguments are evaluated first, then their resulting value is passed into the function. So the function will see [1 2 3].
But macros are not like this, though it is easy to forget. When you call:
(somemacro v)
v is not evaluated and does not pass in [1 2 3]. Inside the macro, all you get is a symbol, v. Now, your macro can emit the symbol you passed in, in the code the macro creates, but it cannot do anything with the value of v unless you add an extra level of evaluation using eval (see footnote -- not recommended).
When you unquote the macro argument, you get a symbol, not a value.
Consider this example:
user> (def a 5)
#'user/a
user> (defmacro m [x] `(+ ~x ~x))
#'user/m
user> (m a)
10
user> (macroexpand '(m a))
(clojure.core/+ a a)
user> (defmacro m [x] `~(+ x x))
#'user/m
user> (m a)
ClassCastException clojure.lang.Symbol cannot be cast to java.lang.Number clojure.lang.Numbers.add (Numbers.java:126)
This would be like trying to do this at the REPL: (+ 'a 'a) Adding symbols, not adding the values those symbols point to.
You might look at these two different definitions of the macro m and think they are doing the same thing. But in the second one, there is an attempt to do something with the symbol x, to do addition with it. In the first macro, the compiler is not asked to do anything with x. It merely is told to emit the addition operation, which will then be actually processed at runtime.
Remember that the point of a macro is to create code not perform runtime activity that a function would do. The code that is created is then immediately run (via implicit eval), so it's easy to forget this separation, but they are two distinct steps.
As a final exercise, pretend you are a compiler. I want you to tell me right now what is the resulting value of this code:
(* t w)
Well? It's impossible. There is no way you can tell me the answer to this question because you have no idea what t and w are. However, later, at runtime, when these presumably have a value, then it will be easy.
And that's what macros do: they output stuff for the runtime to handle.
(Very much not recommended but to help further describe what's happening, this works:)
user> (def a 1)
user> (defmacro m [x] `~(+ (eval x) (eval x)))
user> (m a)
2
I discovered that special forms can not be passed as arguments or saved in variables, both in Clojure:
user=> (defn my-func
[op]
(op 1 2 3))
#'user/my-func
user=> (my-func +)
6
user=> (my-func if)
java.lang.Exception: Unable to resolve symbol: if in this context (NO_SOURCE_FILE:5)
user=> (def my-if if)
java.lang.Exception: Unable to resolve symbol: if in this context (NO_SOURCE_FILE:9)
and in Racket/Scheme:
> (define (my-func op)
(op 1 2 3))
> (my-func +)
6
> (my-func if)
if: bad syntax in: if
> (define my-if if)
*if: bad syntax in: if
That's cool, I'm fine with that, I know that I can just write a function to wrap a special form and then pass that function.
But I'd like to know why Lisps have this restriction, and what negative consequences allowing this would have. Are they any major dialects that do allow that?
It makes evaluation more complicated and compilation hard.
If you have a form (a b c), then you need to resolve at runtime the value of a and then apply it somehow to the forms b and c.
The simpler model of a few limited number of special forms and otherwise strict evaluation is then gone.
See also: FEXPR
Special forms are not functions: functions take values as arguments, whereas special forms take forms. For example, look at your example of if:
(if 1 2 3)
Okay, that's easy, since 2 and 3 are already values. But what about this?
(define (modadd a b n)
(if (zero? n) #f (modulo (+ a b) n)))
In this case, the if is actually receiving the #f and (modulo (+ a b) n) as forms, not as values. And this is important! (modulo x n) will fail if n is 0: that is why it's left unevaluated until we know that n is not 0.
The trouble with being able to pass special forms as first-class objects is that higher-order functions cannot call those objects using predictable semantics: is it a function object, so that you're passing in values, or is it a special form, so that you're passing in forms? This gets to be a huge mess.
Yes, you can write a wrapper function, like my modadd, that encapsulates if. However, there is no way you can just reimplement if as a function, while still preserving the only-evaluate-one-of-the-branches behaviour.