This question builds off of a previous SO question which was for building expressions from expressions inside of of a macro. However, things got a little trucker when quoting the whole expression. For example, I want to build the expression :(name=val). The following:
macro quotetest(name,val)
quote
nm = Meta.quot($(QuoteNode(name)))
v = Meta.quot($(QuoteNode(val)))
println(nm); println(typeof(nm))
println(v); println(typeof(val))
end
end
#quotetest x 5 # Test case: build :(x=5)
prints out
:x
Expr
$(Expr(:quote, 5))
Expr
showing that I am on the right path: nm and val are the expressions that I want inside of the quote. However, I can't seem to apply the previous solution at this point. For example,
macro quotetest(name,val)
quote
nm = Meta.quot($(QuoteNode(name)))
v = Meta.quot($(QuoteNode(val)))
println(nm); println(typeof(nm))
println(v); println(typeof(v))
println(:($(Expr(:(=),$(QuoteNode(nm)),$(QuoteNode(val))))))
end
end
fails, saying nm is not defined. I tried just interpolating without the QuoteNode, escaping the interpolation $(esc(nm)), etc. I can't seem to find out how to make it build the expression.
I think you are using $ signs more than you need to. Is this what you're looking for?
julia> macro quotetest(name,val)
quote
expr = :($$(QuoteNode(name)) = $$(QuoteNode(val)))
println(expr)
display(expr)
println(typeof(expr))
end
end
#quotetest (macro with 1 method)
julia> #quotetest test 1
test = 1
:(test = 1)
Expr
Related
I have been messing around with generated functions in Julia, and have come to a weird problem I do not understand fully: My final goal would involve calling a macro (more specifically #tullio) from within a generated function (to perform some tensor contractions that depend on the input tensors). But I have been having problems, which I narrowed down to calling the macro from within the generated function.
To illustrate the problem, let's consider a very simple example that also fails:
macro my_add(a,b)
return :($a + $b)
end
function add_one_expr(x::T) where T
y = one(T)
return :( #my_add($x,$y) )
end
#generated function add_one_gen(x::T) where T
y = one(T)
return :( #my_add($x,$y) )
end
With these declarations, I find that eval(add_one_expr(2.0)) works just as expected and returns and expression
:(#my_add 2.0 1.0)
which correctly evaluates to 3.0.
However evaluating add_one_gen(2.0) returns the following error:
MethodError: no method matching +(::Type{Float64}, ::Float64)
Doing some research, I have found that #generated actually produces two codes, and in one only the types of the variables can be used. I think this is what is happening here, but I do not understand what is happening at all. It must be some weird interaction between macros and generated functions.
Can someone explain and/or propose a solution? Thank you!
I find it helpful to think of generated functions as having two components: the body and any generated code (the stuff inside a quote..end). The body is evaluated at compile time, and doesn't "know" the values, only the types. So for a generated function taking x::T as an argument, any references to x in the body will actually point to the type T. This can be very confusing. To make things clearer, I recommend the body only refer to types, never to values.
Here's a little example:
julia> #generated function show_val_and_type(x::T) where {T}
quote
println("x is ", x)
println("\$x is ", $x)
println("T is ", T)
println("\$T is ", $T)
end
end
show_val_and_type
julia> show_val_and_type(3)
x is 3
$x is Int64
T is Int64
$T is Int64
The interpolated $x means "take the x from the body (which refers to T) and splice it in.
If you follow the approach of never referring to values in the body, you can test generated functions by removing the #generated, like this:
julia> function add_one_gen(x::T) where T
y = one(T)
quote
#my_add(x,$y)
end
end
add_one_gen
julia> add_one_gen(3)
quote
#= REPL[42]:4 =#
#= REPL[42]:4 =# #my_add x 1
end
That looks reasonable, but when we test it we get
julia> add_one_gen(3)
ERROR: UndefVarError: x not defined
Stacktrace:
[1] macro expansion
# ./REPL[48]:4 [inlined]
[2] add_one_gen(x::Int64)
# Main ./REPL[48]:1
[3] top-level scope
# REPL[49]:1
So let's see what the macro gives us
julia> #macroexpand #my_add x 1
:(Main.x + 1)
It's pointing to Main.x, which doesn't exist. The macro is being too eager, and we need to delay its evaluation. The standard way to do this is with esc. So finally, this works:
julia> macro my_add(a,b)
return :($(esc(a)) + $(esc(b)))
end
#my_add
julia> #generated function add_one_gen(x::T) where T
y = one(T)
quote
#my_add(x,$y)
end
end
add_one_gen
julia> add_one_gen(3)
4
All these following lines of code are Julia expressions:
x = 10
1 + 1
println("hi")
if you want to pass an expression to a macro, it works like this. Macro foo just returns the given expression, which will be executed:
macro foo(ex)
return ex
end
#foo println("yes") # prints yes
x = #foo 1+1
println(x) # prints 2
If you want to convert a string into an expression, you can use Meta.parse():
string = "1+1"
expr = Meta.parse(string)
x = #foo expr
println(x) # prints 1 + 1
But, obviously, the macro treats expr as a symbol. What am i getting wrong here?
Thanks in advance!
Macro hygiene is important "macros must ensure that the variables they introduce in their returned expressions do not accidentally clash with existing variables in the surrounding code they expand into." There is a section in the docs. It is easiest just to show a simple case:
macro foo(x)
return :($x)
end
When you enter an ordinary expression in the REPL, it is evaluated immediately. To suppress that evaluation, surround the expression with :( ).
julia> 1 + 1
2
julia> :(1 + 1)
:(1 + 1)
# note this is the same result as you get using Meta.parse
julia> Meta.parse("1 + 1")
:(1 + 1)
So, Meta.parse will convert an appropriate string to an expression. And if you eval the result, the expression will be evaluated. Note that printing a simple expression removes the outer :( )
julia> expr = Meta.parse("1 + 1")
:(1 + 1)
julia> print(expr)
1 + 1
julia> result = eval(expr)
2
Usually, macros are used to manipulate things before the usual evaluation of expressions; they are syntax transformations, mostly. Macros are performed before other source code is compiled/evaluated/executed.
Rather than seeking a macro that evaluates a string as if it were typed directly into the REPL (without quotes), use this function instead.
evalstr(x::AbstractString) = eval(Meta.parse(x))
While I do not recommend this next macro, it is good to know the technique.
A macro named <name>_str is used like this <name>"<string contents>" :
julia> macro eval_str(x)
:(eval(Meta.parse($x)))
end
julia> eval"1 + 1"
2
(p.s. do not reuse Base function names as variable names, use str not string)
Please let me know if there is something I have not addressed.
Consider these two macro definitions:
macro createTest1()
quote
function test(a = false)
a
end
end |> esc
end
macro createTest2()
args = :(a = false)
quote
function test($args)
a
end
end |> esc
end
According to the builtin Julia facilities they should both evaluate to the same thing when expanded:
println(#macroexpand #createTest1)
begin
function test(a=false)
a
end
end
println(#macroexpand #createTest2)
begin
function test(a = false)
a
end
end
Still I get a parse error when trying to evaluate the second macro:
#createTest2
ERROR: LoadError: syntax: "a = false" is not a valid function argument name
It is a space in the second argument list. However, that should be correct Julia syntax. My guess is that it interprets the second argument list as another Julia construct compared to the first. If that is the case how do I get around it?
The reason that the second macro is failing as stated in my question above. It looks correct when printed however args is not defined correctly and Julia interprets it as an expression which is not allowed. The solution is to instead define args according to the rules for function parameters. The following code executes as expected:
macro createTest2()
args = Expr(:kw, :x, false)
quote
function test($(args))
a
end
end |> esc
end
I am a noob at metaprogramming so maybe I am not understanding this. I thought the purpose of the #nloops macro in Base.Cartesian was to make it possible to code an arbitrary number of nested for loops, in circumstances where the dimension is unknown a priori. In the documentation for the module, the following example is given:
#nloops 3 i A begin
s += #nref 3 A i
end
which evaluates to
for i_3 = 1:size(A,3)
for i_2 = 1:size(A,2)
for i_1 = 1:size(A,1)
s += A[i_1,i_2,i_3]
end
end
end
Here, the number 3 is known a priori. For my purposes, however, and for the purposes that I thought nloops was created, the number of nested levels is not known ahead of time. So I would not be able to hard code the integer 3. Even in the documentation, it is stated:
The (basic) syntax of #nloops is as follows:
The first argument must be an integer (not a variable) specifying the number of loops.
...
If I assign an integer value - say the dimension of an array that is passed to a function - to some variable, the nloops macro no longer works:
b = 3
#nloops b i A begin
s += #nref b A i
end
This returns an error:
ERROR: LoadError: MethodError: no method matching _nloops(::Symbol, ::Symbol, ::Symbol, ::Expr)
Closest candidates are:
_nloops(::Int64, ::Symbol, ::Symbol, ::Expr...) at cartesian.jl:43
...
I don't know how to have nloops evaluate the b variable as an integer rather than a symbol. I have looked at the documentation and tried various iterations of eval and other functions and macros but it is either interpreted as a symbol or an Expr. What is the correct, julian way to write this?
See supplying the number of expressions:
julia> A = rand(4, 4, 3) # 3D array (Array{Int, 3})
A generated function is kinda like a macro, in that the resulting expression is not returned, but compiled and executed on invocation/call, it also sees the type (and their type parameters of course) of the arguments, ie:
inside the generated function, A is Array{T, N}, not the value of the array.
so T is Int and N is 3!
Here inside the quoted expression, N is interpolated into the expression, with the syntax $N, which evaluates to 3:
julia> #generated function mysum(A::Array{T,N}) where {T,N}
quote
s = zero(T)
#nloops $N i A begin
s += #nref $N A i
end
s
end
end
mysum (generic function with 1 method)
julia> mysum(A)
23.2791638775186
You could construct the expression and then evaluate it, ie.:
julia> s = 0; n = 3;
julia> _3loops = quote
#nloops $n i A begin
global s += #nref $n A i
end
end
quote
#nloops 3 i A begin
global s += #nref(3, A, i)
end
end
julia> eval(_3loops)
julia> s
23.2791638775186
I have scrubbed manually the LineNumberNodes from the AST for readability (there is also MacroTools.prettify, that does it for you).
Running this example in the REPL needs to declare s as global inside the loop in Julia 1.0.
I'd like to write a simple macro that shows the names & values of variables. In Common Lisp it would be
(defmacro dprint (&rest vars)
`(progn
,#(loop for v in vars
collect `(format t "~a: ~a~%" ',v ,v))))
In Julia I had two problems writing this:
How can I collect the generated Expr objects into a block? (In Lisp, this is done by splicing the list with ,# into progn.) The best I could come up with is to create an Expr(:block), and set its args to the list, but this is far from elegant.
I need to use both the name and the value of the variable. Interpolation inside strings and quoted expressions both use $, which complicates the issue, but even if I use string for concatenation, I can 't print the variable's name - at least :($v) does not do the same as ',v in CL...
My current macro looks like this:
macro dprint(vars...)
ex = Expr(:block)
ex.args = [:(println(string(:($v), " = ", $v))) for v in vars]
ex
end
Looking at a macroexpansion shows the problem:
julia> macroexpand(:(#dprint x y))
quote
println(string(v," = ",x))
println(string(v," = ",y))
end
I would like to get
quote
println(string(:x," = ",x))
println(string(:y," = ",y))
end
Any hints?
EDIT: Combining the answers, the solution seems to be the following:
macro dprint(vars...)
quote
$([:(println(string($(Meta.quot(v)), " = ", $v))) for v in vars]...)
end
end
... i.e., using $(Meta.quot(v)) to the effect of ',v, and $(expr...) for ,#expr. Thank you again!
the #show macro already exists for this. It is helpful to be able to implement it yourself, so later you can do other likes like make one that will show the size of an Array..
For your particular variant:
Answer is Meta.quot,
macro dprint(vars...)
ex = Expr(:block)
ex.args = [:(println($(Meta.quot(v)), " = ", $v)) for v in vars]
ex
end
See with:
julia> a=2; b=3;
julia> #dprint a
a = 2
julia> #dprint a b
a = 2
b = 3
oxinabox's answer is good, but I should mention the equivalent to ,#x is $(x...) (this is the other part of your question).
For instance, consider the macro
macro _begin(); esc(:begin); end
macro #_begin()(args...)
quote
$(args...)
end |> esc
end
and invocation
#begin x=1 y=2 x*y
which (though dubiously readable) produces the expected result 2. (The #_begin macro is not part of the example; it is required however because begin is a reserved word, so one needs a macro to access the symbol directly.)
Note
julia> macroexpand(:(#begin 1 2 3))
quote # REPL[1], line 5:
1
2
3
end
I consider this more readable, personally, than pushing to the .args array.