Consider these two macro definitions:
macro createTest1()
quote
function test(a = false)
a
end
end |> esc
end
macro createTest2()
args = :(a = false)
quote
function test($args)
a
end
end |> esc
end
According to the builtin Julia facilities they should both evaluate to the same thing when expanded:
println(#macroexpand #createTest1)
begin
function test(a=false)
a
end
end
println(#macroexpand #createTest2)
begin
function test(a = false)
a
end
end
Still I get a parse error when trying to evaluate the second macro:
#createTest2
ERROR: LoadError: syntax: "a = false" is not a valid function argument name
It is a space in the second argument list. However, that should be correct Julia syntax. My guess is that it interprets the second argument list as another Julia construct compared to the first. If that is the case how do I get around it?
The reason that the second macro is failing as stated in my question above. It looks correct when printed however args is not defined correctly and Julia interprets it as an expression which is not allowed. The solution is to instead define args according to the rules for function parameters. The following code executes as expected:
macro createTest2()
args = Expr(:kw, :x, false)
quote
function test($(args))
a
end
end |> esc
end
Related
I have been messing around with generated functions in Julia, and have come to a weird problem I do not understand fully: My final goal would involve calling a macro (more specifically #tullio) from within a generated function (to perform some tensor contractions that depend on the input tensors). But I have been having problems, which I narrowed down to calling the macro from within the generated function.
To illustrate the problem, let's consider a very simple example that also fails:
macro my_add(a,b)
return :($a + $b)
end
function add_one_expr(x::T) where T
y = one(T)
return :( #my_add($x,$y) )
end
#generated function add_one_gen(x::T) where T
y = one(T)
return :( #my_add($x,$y) )
end
With these declarations, I find that eval(add_one_expr(2.0)) works just as expected and returns and expression
:(#my_add 2.0 1.0)
which correctly evaluates to 3.0.
However evaluating add_one_gen(2.0) returns the following error:
MethodError: no method matching +(::Type{Float64}, ::Float64)
Doing some research, I have found that #generated actually produces two codes, and in one only the types of the variables can be used. I think this is what is happening here, but I do not understand what is happening at all. It must be some weird interaction between macros and generated functions.
Can someone explain and/or propose a solution? Thank you!
I find it helpful to think of generated functions as having two components: the body and any generated code (the stuff inside a quote..end). The body is evaluated at compile time, and doesn't "know" the values, only the types. So for a generated function taking x::T as an argument, any references to x in the body will actually point to the type T. This can be very confusing. To make things clearer, I recommend the body only refer to types, never to values.
Here's a little example:
julia> #generated function show_val_and_type(x::T) where {T}
quote
println("x is ", x)
println("\$x is ", $x)
println("T is ", T)
println("\$T is ", $T)
end
end
show_val_and_type
julia> show_val_and_type(3)
x is 3
$x is Int64
T is Int64
$T is Int64
The interpolated $x means "take the x from the body (which refers to T) and splice it in.
If you follow the approach of never referring to values in the body, you can test generated functions by removing the #generated, like this:
julia> function add_one_gen(x::T) where T
y = one(T)
quote
#my_add(x,$y)
end
end
add_one_gen
julia> add_one_gen(3)
quote
#= REPL[42]:4 =#
#= REPL[42]:4 =# #my_add x 1
end
That looks reasonable, but when we test it we get
julia> add_one_gen(3)
ERROR: UndefVarError: x not defined
Stacktrace:
[1] macro expansion
# ./REPL[48]:4 [inlined]
[2] add_one_gen(x::Int64)
# Main ./REPL[48]:1
[3] top-level scope
# REPL[49]:1
So let's see what the macro gives us
julia> #macroexpand #my_add x 1
:(Main.x + 1)
It's pointing to Main.x, which doesn't exist. The macro is being too eager, and we need to delay its evaluation. The standard way to do this is with esc. So finally, this works:
julia> macro my_add(a,b)
return :($(esc(a)) + $(esc(b)))
end
#my_add
julia> #generated function add_one_gen(x::T) where T
y = one(T)
quote
#my_add(x,$y)
end
end
add_one_gen
julia> add_one_gen(3)
4
I have a macro which gets a module name as parameter and I want to call a function on that module to get some data in order to generate the quote block.
Example:
defmacro my_macro(module) do
data = apply(module, :config, [])
# do something with data to generate the quote do end
end
Obviously, this doesn't work because the parameter value is quoted. I could fetch the data inside the quote block and act accordingly but that would put the whole logic inside the module that uses my macro which is quite dirty. I want to inject as little code as possible.
You can extract the module out by pattern matching with its quoted form: {:__aliases__, _, list} where list is a list of atoms which when concatenated with a dot (use Module.concat/1) produces the full module name.
defmodule A do
defmacro my_macro({:__aliases__, _, list}) do
module = Module.concat(list)
module.foo()
end
end
defmodule B do
def foo do
quote do
42
end
end
end
defmodule C do
import A
IO.inspect my_macro B
end
Output:
42
This question builds off of a previous SO question which was for building expressions from expressions inside of of a macro. However, things got a little trucker when quoting the whole expression. For example, I want to build the expression :(name=val). The following:
macro quotetest(name,val)
quote
nm = Meta.quot($(QuoteNode(name)))
v = Meta.quot($(QuoteNode(val)))
println(nm); println(typeof(nm))
println(v); println(typeof(val))
end
end
#quotetest x 5 # Test case: build :(x=5)
prints out
:x
Expr
$(Expr(:quote, 5))
Expr
showing that I am on the right path: nm and val are the expressions that I want inside of the quote. However, I can't seem to apply the previous solution at this point. For example,
macro quotetest(name,val)
quote
nm = Meta.quot($(QuoteNode(name)))
v = Meta.quot($(QuoteNode(val)))
println(nm); println(typeof(nm))
println(v); println(typeof(v))
println(:($(Expr(:(=),$(QuoteNode(nm)),$(QuoteNode(val))))))
end
end
fails, saying nm is not defined. I tried just interpolating without the QuoteNode, escaping the interpolation $(esc(nm)), etc. I can't seem to find out how to make it build the expression.
I think you are using $ signs more than you need to. Is this what you're looking for?
julia> macro quotetest(name,val)
quote
expr = :($$(QuoteNode(name)) = $$(QuoteNode(val)))
println(expr)
display(expr)
println(typeof(expr))
end
end
#quotetest (macro with 1 method)
julia> #quotetest test 1
test = 1
:(test = 1)
Expr
I would like to inject code into a function. For concreteness, consider a simple simulater:
function simulation(A, x)
for t in 1:1000
z = randn(3)
x = A*x + z
end
end
Sometimes I would like to record the values of x every ten time-steps, sometimes the values of z every 20 time-steps, and sometimes I don't want to record any values. I could, of course, put some flags as arguments to the function, and have some if-else statements. But I would like to rather keep the simulation code clean, and only inject a piece of code like
if t%10 == 0
append!(rec_z, z)
end
into particular places of the function whenever I need it. For that, I'd like to write a macro such that monitoring a particular value becomes
#monitor(:z, 10)
simulation(A, x)
Is that possible with Julia's Metaprogramming capabilities?
No, you cannot use metaprogramming to inject code into an already-written function. Metaprogramming can only do things that you could directly write yourself at precisely the location where the macro itself is written. That means that a statement like:
#monitor(:z, 10); simulation(A, x)
cannot even modify the simulation(A, x) function call. It can only expand out to some normal Julia code that runs before simulation is called. You could, perhaps, include the simulation function call as an argument to the macro, e.g., #monitor(:z, 10, simulation(A, x)), but now all the macro can do is change the function call itself. It still cannot "go back" and add new code to a function that was already written.
You could, however, carefully and meticulously craft a macro that takes the function definition body and modifies it to add your debug code, e.g.,
#monitor(:z, 10, function simulation(A, x)
for t in 1:1000
# ...
end
end)
But now you must write code in the macro that traverses the code in the function body, and injects your debug statement at the correct place. This is not an easy task. And it's even harder to write in a robust manner that wouldn't break the moment you modified your actual simulation code.
Traversing code and inserting it is a much easier task for you to do yourself with an editor. A common idiom for debugging statements is to use a one-liner, like this:
const debug = false
function simulation (A, x)
for t in 1:1000
z = rand(3)
x = A*x + z
debug && t%10==0 && append!(rec_z, z)
end
end
What's really cool here is that by marking debug as constant, Julia is able to completely optimize away the debugging code when it's false — it doesn't even appear in the generated code! So there is no overhead when you're not debugging. It does mean, however, that you have to restart Julia (or reload the module it's in) for you to change the debug flag. Even when debug isn't marked as const, I cannot measure any overhead for this simple loop. And chances are, your loop will be more complicated than this one. So don't worry about performance here until you actually double-check that it's having an effect.
You might be interested in this which i just whipped up. It doesn't QUITE do what you are doing, but it's close. Generally safe and consistent places to add code are the beginning and end of code blocks. These macros allow you to inject some code in those location (and even pass code parameters!)
Should be useful for say toggle-able input checking.
#cleaninject.jl
#cleanly injects some code into the AST of a function.
function code_to_inject()
println("this code is injected")
end
function code_to_inject(a,b)
println("injected code handles $a and $b")
end
macro inject_code_prepend(f)
#make sure this macro precedes a function definition.
isa(f, Expr) || error("checkable macro must precede a function definition")
(f.head == :function) || error("checkable macro must precede a function definition")
#be lazy and let the parser do the hard work.
b2 = parse("code_to_inject()")
#inject the generated code into the AST.
unshift!(f.args[2].args, b2)
#return the escaped function to the parser so that it generates the new function.
return Expr(:escape, f)
end
macro inject_code_append(f)
#make sure this macro precedes a function definition.
isa(f, Expr) || error("checkable macro must precede a function definition")
(f.head == :function) || error("checkable macro must precede a function definition")
#be lazy and let the parser do the hard work.
b2 = parse("code_to_inject()")
#inject the generated code into the AST.
push!(f.args[2].args, b2)
#return the escaped function to the parser so that it generates the new function.
return Expr(:escape, f)
end
macro inject_code_with_args(f)
#make sure this macro precedes a function definition.
isa(f, Expr) || error("checkable macro must precede a function definition")
(f.head == :function) || error("checkable macro must precede a function definition")
#be lazy and let the parser do the hard work.
b2 = parse(string("code_to_inject(", join(f.args[1].args[2:end], ","), ")"))
#inject the generated code into the AST.
unshift!(f.args[2].args, b2)
#return the escaped function to the parser so that it generates the new function.
return Expr(:escape, f)
end
################################################################################
# RESULTS
#=
julia> #inject_code_prepend function p()
println("victim function")
end
p (generic function with 1 method)
julia> p()
this code is injected
victim function
julia> #inject_code_append function p()
println("victim function")
end
p (generic function with 1 method)
julia> p()
victim function
this code is injected
julia> #inject_code_with_args function p(a, b)
println("victim called with $a and $b")
end
p (generic function with 2 methods)
julia> p(1, 2)
injected code handles 1 and 2
victim called with 1 and 2
=#
I am diving into Metaprogramming Elixir by Chris McCord.
I made a spelling mistake while typing one of examples:
defmodule Math do
defmacro say({:+, _, [lhs, rhs]}) do
qoute do #spelling mistake (swapped "o" and "u")
lhs = unquote(lhs) #(CompileError) math.exs:4: undefined function lhs/0
rhs = unquote(rhs)
result = lhs + rhs
IO.puts "#{lhs} plus #{rhs} is #{result}"
result
end
end
defmacro say({:*, _, [lhs, rhs]}) do
qoute do #spelling mistake copied
lhs = unquote(lhs)
rhs = unquote(rhs)
result = lhs * rhs
IO.puts "#{lhs} times #{rhs} is #{result}"
result
end
end
end
In the shell, the errors are meaningful:
iex(1)> qoute do: 1 + 2 #spelling mistake
** (RuntimeError) undefined function: qoute/1
iex(1)> unquote do: 1
** (CompileError) iex:1: unquote called outside quote
Why compiling this file gives error in the next line? Is my spelling mistake some valid construct?
The error appears in correct file, if I remove the unquote.
defmodule Math do
defmacro say({:+, _, [lhs, rhs]}) do
qoute do #function qoute/1 undefined
result = lhs + rhs
IO.puts "#{lhs} plus #{rhs} is #{result}"
result
end
end
...
Why using unquote moved the error somewhere else?
That's because once you call qoute/1, Elixir assumes it is a function that will be defined later, and proceeds to compile the code as a function call. However, when we try to compile it, we see an unquote, assume there is a variable defined outside, and everything crashes when it doesn't.
There is no way we can work around it because the error happens when we are expanding code and is exactly when quote/unquote are expanded too.