I would have thought this would work:
macro meta_meta(x,y)
:(macro $x(arg) :($($y) + $arg) end)
end
The expected behavior is that calling #meta_meta(f,2) should be equivalent to macro f(arg) :(2 + $arg) end
In other words:
julia> #meta_meta(f,2)
julia> #f(3)
5
Instead I get:
ERROR: syntax: invalid macro definition
I'm at a bit of a loss for how to proceed. I see that the expression tree for this macro is different from the one I get if I manually generate #f and examine its expression tree, and I've tried several iterations of #meta_meta, but I cannot figure out how to change my definition to get it working.
Macro hygiene is a little finnicky when dealing with a quote inside a quote. Often I find the only way is to refuse macro hygiene entirely, and use gensym liberally to simulate it.
However in your reduced example, it's straightforward to just turn the inner quote into an Expr:
julia> macro meta_meta(x, y)
:(macro $(esc(x))(arg) Expr(:call, :+, $(esc(y)), esc(arg)) end)
end
#meta_meta (macro with 1 method)
julia> #meta_meta f 2
#f (macro with 1 method)
julia> #f 3
5
If things get more complicated, the approach I mentioned above involves turning off macro hygiene with esc. This means that we have to do the hygiene ourself, hence the gensym:
julia> macro meta_meta(x, y)
arg = gensym()
esc(:(macro $x($arg) :($$y + $$arg) end))
end
#meta_meta (macro with 1 method)
julia> #meta_meta f 2
#f (macro with 1 method)
julia> #f 3
5
Related
I have been messing around with generated functions in Julia, and have come to a weird problem I do not understand fully: My final goal would involve calling a macro (more specifically #tullio) from within a generated function (to perform some tensor contractions that depend on the input tensors). But I have been having problems, which I narrowed down to calling the macro from within the generated function.
To illustrate the problem, let's consider a very simple example that also fails:
macro my_add(a,b)
return :($a + $b)
end
function add_one_expr(x::T) where T
y = one(T)
return :( #my_add($x,$y) )
end
#generated function add_one_gen(x::T) where T
y = one(T)
return :( #my_add($x,$y) )
end
With these declarations, I find that eval(add_one_expr(2.0)) works just as expected and returns and expression
:(#my_add 2.0 1.0)
which correctly evaluates to 3.0.
However evaluating add_one_gen(2.0) returns the following error:
MethodError: no method matching +(::Type{Float64}, ::Float64)
Doing some research, I have found that #generated actually produces two codes, and in one only the types of the variables can be used. I think this is what is happening here, but I do not understand what is happening at all. It must be some weird interaction between macros and generated functions.
Can someone explain and/or propose a solution? Thank you!
I find it helpful to think of generated functions as having two components: the body and any generated code (the stuff inside a quote..end). The body is evaluated at compile time, and doesn't "know" the values, only the types. So for a generated function taking x::T as an argument, any references to x in the body will actually point to the type T. This can be very confusing. To make things clearer, I recommend the body only refer to types, never to values.
Here's a little example:
julia> #generated function show_val_and_type(x::T) where {T}
quote
println("x is ", x)
println("\$x is ", $x)
println("T is ", T)
println("\$T is ", $T)
end
end
show_val_and_type
julia> show_val_and_type(3)
x is 3
$x is Int64
T is Int64
$T is Int64
The interpolated $x means "take the x from the body (which refers to T) and splice it in.
If you follow the approach of never referring to values in the body, you can test generated functions by removing the #generated, like this:
julia> function add_one_gen(x::T) where T
y = one(T)
quote
#my_add(x,$y)
end
end
add_one_gen
julia> add_one_gen(3)
quote
#= REPL[42]:4 =#
#= REPL[42]:4 =# #my_add x 1
end
That looks reasonable, but when we test it we get
julia> add_one_gen(3)
ERROR: UndefVarError: x not defined
Stacktrace:
[1] macro expansion
# ./REPL[48]:4 [inlined]
[2] add_one_gen(x::Int64)
# Main ./REPL[48]:1
[3] top-level scope
# REPL[49]:1
So let's see what the macro gives us
julia> #macroexpand #my_add x 1
:(Main.x + 1)
It's pointing to Main.x, which doesn't exist. The macro is being too eager, and we need to delay its evaluation. The standard way to do this is with esc. So finally, this works:
julia> macro my_add(a,b)
return :($(esc(a)) + $(esc(b)))
end
#my_add
julia> #generated function add_one_gen(x::T) where T
y = one(T)
quote
#my_add(x,$y)
end
end
add_one_gen
julia> add_one_gen(3)
4
All these following lines of code are Julia expressions:
x = 10
1 + 1
println("hi")
if you want to pass an expression to a macro, it works like this. Macro foo just returns the given expression, which will be executed:
macro foo(ex)
return ex
end
#foo println("yes") # prints yes
x = #foo 1+1
println(x) # prints 2
If you want to convert a string into an expression, you can use Meta.parse():
string = "1+1"
expr = Meta.parse(string)
x = #foo expr
println(x) # prints 1 + 1
But, obviously, the macro treats expr as a symbol. What am i getting wrong here?
Thanks in advance!
Macro hygiene is important "macros must ensure that the variables they introduce in their returned expressions do not accidentally clash with existing variables in the surrounding code they expand into." There is a section in the docs. It is easiest just to show a simple case:
macro foo(x)
return :($x)
end
When you enter an ordinary expression in the REPL, it is evaluated immediately. To suppress that evaluation, surround the expression with :( ).
julia> 1 + 1
2
julia> :(1 + 1)
:(1 + 1)
# note this is the same result as you get using Meta.parse
julia> Meta.parse("1 + 1")
:(1 + 1)
So, Meta.parse will convert an appropriate string to an expression. And if you eval the result, the expression will be evaluated. Note that printing a simple expression removes the outer :( )
julia> expr = Meta.parse("1 + 1")
:(1 + 1)
julia> print(expr)
1 + 1
julia> result = eval(expr)
2
Usually, macros are used to manipulate things before the usual evaluation of expressions; they are syntax transformations, mostly. Macros are performed before other source code is compiled/evaluated/executed.
Rather than seeking a macro that evaluates a string as if it were typed directly into the REPL (without quotes), use this function instead.
evalstr(x::AbstractString) = eval(Meta.parse(x))
While I do not recommend this next macro, it is good to know the technique.
A macro named <name>_str is used like this <name>"<string contents>" :
julia> macro eval_str(x)
:(eval(Meta.parse($x)))
end
julia> eval"1 + 1"
2
(p.s. do not reuse Base function names as variable names, use str not string)
Please let me know if there is something I have not addressed.
When I try the following code snippet, I got that the variable i is not found. Why is that ?
function evalMyExpr(expr,n)
for i in 1:n
eval(expr)
end
end
expr1 = Meta.parse("println(\"hello\")")
expr2 = Meta.parse("println(string(i))")
evalMyExpr(expr1,2) # ok
evalMyExpr(expr2,2) # UndefVarError: i not defined
Note that if I transform it in a macro it works:
macro evalMyExprMacro(expr,n)
quote
for i in 1:$n
$expr
end
end
end
#evalMyExprMacro println(string(i)) 2 # ok
More in general, which is the difference between a function that accepts expressions as parameters and a macro?
Expressions passed to functions are just normal values that are processed at run time. The reason why code fails when it is passed expr2 is because eval evaluates expressions in global scope (in general it is not recommended to use eval in functions). Therefore, as probably variable i is not defined in global scope in your case, you get an error. See an example when i is defined in global scope:
julia> i = 1000
1000
julia> function evalMyExpr(expr,n)
for i in 1:n
eval(expr)
end
end
evalMyExpr (generic function with 1 method)
julia> expr2 = Meta.parse("println(string(i))")
:(println(string(i)))
julia>
julia> evalMyExpr(expr2,2)
1000
1000
Now - in marcos expressions are processed in compile time (before the code is run) so the expression you use is injected into the code generated by the macro that is executed afterwards. You can see the effect by using #macroexpand:
julia> macro evalMyExprMacro(expr,n)
quote
for i in 1:$n
$expr
end
end
end
#evalMyExprMacro (macro with 1 method)
julia> #macroexpand #evalMyExprMacro println(string(i)) 2
quote
#= REPL[23]:3 =#
for #6#i = 1:2
#= REPL[23]:4 =#
(Main.println)((Main.string)(#6#i))
end
end
Observe that the variable name was changed by the macro processing mechanizm to #6#i and it matches the name of the variable that is used in the for loop.
I'd like to write a simple macro that shows the names & values of variables. In Common Lisp it would be
(defmacro dprint (&rest vars)
`(progn
,#(loop for v in vars
collect `(format t "~a: ~a~%" ',v ,v))))
In Julia I had two problems writing this:
How can I collect the generated Expr objects into a block? (In Lisp, this is done by splicing the list with ,# into progn.) The best I could come up with is to create an Expr(:block), and set its args to the list, but this is far from elegant.
I need to use both the name and the value of the variable. Interpolation inside strings and quoted expressions both use $, which complicates the issue, but even if I use string for concatenation, I can 't print the variable's name - at least :($v) does not do the same as ',v in CL...
My current macro looks like this:
macro dprint(vars...)
ex = Expr(:block)
ex.args = [:(println(string(:($v), " = ", $v))) for v in vars]
ex
end
Looking at a macroexpansion shows the problem:
julia> macroexpand(:(#dprint x y))
quote
println(string(v," = ",x))
println(string(v," = ",y))
end
I would like to get
quote
println(string(:x," = ",x))
println(string(:y," = ",y))
end
Any hints?
EDIT: Combining the answers, the solution seems to be the following:
macro dprint(vars...)
quote
$([:(println(string($(Meta.quot(v)), " = ", $v))) for v in vars]...)
end
end
... i.e., using $(Meta.quot(v)) to the effect of ',v, and $(expr...) for ,#expr. Thank you again!
the #show macro already exists for this. It is helpful to be able to implement it yourself, so later you can do other likes like make one that will show the size of an Array..
For your particular variant:
Answer is Meta.quot,
macro dprint(vars...)
ex = Expr(:block)
ex.args = [:(println($(Meta.quot(v)), " = ", $v)) for v in vars]
ex
end
See with:
julia> a=2; b=3;
julia> #dprint a
a = 2
julia> #dprint a b
a = 2
b = 3
oxinabox's answer is good, but I should mention the equivalent to ,#x is $(x...) (this is the other part of your question).
For instance, consider the macro
macro _begin(); esc(:begin); end
macro #_begin()(args...)
quote
$(args...)
end |> esc
end
and invocation
#begin x=1 y=2 x*y
which (though dubiously readable) produces the expected result 2. (The #_begin macro is not part of the example; it is required however because begin is a reserved word, so one needs a macro to access the symbol directly.)
Note
julia> macroexpand(:(#begin 1 2 3))
quote # REPL[1], line 5:
1
2
3
end
I consider this more readable, personally, than pushing to the .args array.
This question builds off of a previous SO question which was for building expressions from expressions inside of of a macro. However, things got a little trucker when quoting the whole expression. For example, I want to build the expression :(name=val). The following:
macro quotetest(name,val)
quote
nm = Meta.quot($(QuoteNode(name)))
v = Meta.quot($(QuoteNode(val)))
println(nm); println(typeof(nm))
println(v); println(typeof(val))
end
end
#quotetest x 5 # Test case: build :(x=5)
prints out
:x
Expr
$(Expr(:quote, 5))
Expr
showing that I am on the right path: nm and val are the expressions that I want inside of the quote. However, I can't seem to apply the previous solution at this point. For example,
macro quotetest(name,val)
quote
nm = Meta.quot($(QuoteNode(name)))
v = Meta.quot($(QuoteNode(val)))
println(nm); println(typeof(nm))
println(v); println(typeof(v))
println(:($(Expr(:(=),$(QuoteNode(nm)),$(QuoteNode(val))))))
end
end
fails, saying nm is not defined. I tried just interpolating without the QuoteNode, escaping the interpolation $(esc(nm)), etc. I can't seem to find out how to make it build the expression.
I think you are using $ signs more than you need to. Is this what you're looking for?
julia> macro quotetest(name,val)
quote
expr = :($$(QuoteNode(name)) = $$(QuoteNode(val)))
println(expr)
display(expr)
println(typeof(expr))
end
end
#quotetest (macro with 1 method)
julia> #quotetest test 1
test = 1
:(test = 1)
Expr