I have this macro:
macro_rules! set_vars {
( $($x:ident),* ) => {
let outer = 42;
$( let $x = outer; )*
}
}
Which expands this invocation:
set_vars!(x, y, z);
into what I expect (from --pretty=expanded):
let outer = 42;
let x = outer;
let y = outer;
let z = outer;
In the subsequent code I can print x, y, and z just fine, but outer seems to be undefined:
error[E0425]: cannot find value `outer` in this scope
--> src/main.rs:11:5
|
11 | outer;
| ^^^^^ not found in this scope
I can access the outer variable if I pass it as an explicit macro parameter.
Is this intentional, something to do with "macro hygiene"? If so, then it would probably make sense to mark such "internal" variables in --pretty=expanded in some special way?
Yes, this is macro hygiene. Identifiers declared within the macro are not available outside of the macro (and vice versa). Rust macros are not C macros (that is, Rust macros are more than glorified text replacement).
See also:
The Little Book of Rust Macros
A Practical Intro to Macros
So, what are hygienic macros anyway?
Related
I have been messing around with generated functions in Julia, and have come to a weird problem I do not understand fully: My final goal would involve calling a macro (more specifically #tullio) from within a generated function (to perform some tensor contractions that depend on the input tensors). But I have been having problems, which I narrowed down to calling the macro from within the generated function.
To illustrate the problem, let's consider a very simple example that also fails:
macro my_add(a,b)
return :($a + $b)
end
function add_one_expr(x::T) where T
y = one(T)
return :( #my_add($x,$y) )
end
#generated function add_one_gen(x::T) where T
y = one(T)
return :( #my_add($x,$y) )
end
With these declarations, I find that eval(add_one_expr(2.0)) works just as expected and returns and expression
:(#my_add 2.0 1.0)
which correctly evaluates to 3.0.
However evaluating add_one_gen(2.0) returns the following error:
MethodError: no method matching +(::Type{Float64}, ::Float64)
Doing some research, I have found that #generated actually produces two codes, and in one only the types of the variables can be used. I think this is what is happening here, but I do not understand what is happening at all. It must be some weird interaction between macros and generated functions.
Can someone explain and/or propose a solution? Thank you!
I find it helpful to think of generated functions as having two components: the body and any generated code (the stuff inside a quote..end). The body is evaluated at compile time, and doesn't "know" the values, only the types. So for a generated function taking x::T as an argument, any references to x in the body will actually point to the type T. This can be very confusing. To make things clearer, I recommend the body only refer to types, never to values.
Here's a little example:
julia> #generated function show_val_and_type(x::T) where {T}
quote
println("x is ", x)
println("\$x is ", $x)
println("T is ", T)
println("\$T is ", $T)
end
end
show_val_and_type
julia> show_val_and_type(3)
x is 3
$x is Int64
T is Int64
$T is Int64
The interpolated $x means "take the x from the body (which refers to T) and splice it in.
If you follow the approach of never referring to values in the body, you can test generated functions by removing the #generated, like this:
julia> function add_one_gen(x::T) where T
y = one(T)
quote
#my_add(x,$y)
end
end
add_one_gen
julia> add_one_gen(3)
quote
#= REPL[42]:4 =#
#= REPL[42]:4 =# #my_add x 1
end
That looks reasonable, but when we test it we get
julia> add_one_gen(3)
ERROR: UndefVarError: x not defined
Stacktrace:
[1] macro expansion
# ./REPL[48]:4 [inlined]
[2] add_one_gen(x::Int64)
# Main ./REPL[48]:1
[3] top-level scope
# REPL[49]:1
So let's see what the macro gives us
julia> #macroexpand #my_add x 1
:(Main.x + 1)
It's pointing to Main.x, which doesn't exist. The macro is being too eager, and we need to delay its evaluation. The standard way to do this is with esc. So finally, this works:
julia> macro my_add(a,b)
return :($(esc(a)) + $(esc(b)))
end
#my_add
julia> #generated function add_one_gen(x::T) where T
y = one(T)
quote
#my_add(x,$y)
end
end
add_one_gen
julia> add_one_gen(3)
4
All these following lines of code are Julia expressions:
x = 10
1 + 1
println("hi")
if you want to pass an expression to a macro, it works like this. Macro foo just returns the given expression, which will be executed:
macro foo(ex)
return ex
end
#foo println("yes") # prints yes
x = #foo 1+1
println(x) # prints 2
If you want to convert a string into an expression, you can use Meta.parse():
string = "1+1"
expr = Meta.parse(string)
x = #foo expr
println(x) # prints 1 + 1
But, obviously, the macro treats expr as a symbol. What am i getting wrong here?
Thanks in advance!
Macro hygiene is important "macros must ensure that the variables they introduce in their returned expressions do not accidentally clash with existing variables in the surrounding code they expand into." There is a section in the docs. It is easiest just to show a simple case:
macro foo(x)
return :($x)
end
When you enter an ordinary expression in the REPL, it is evaluated immediately. To suppress that evaluation, surround the expression with :( ).
julia> 1 + 1
2
julia> :(1 + 1)
:(1 + 1)
# note this is the same result as you get using Meta.parse
julia> Meta.parse("1 + 1")
:(1 + 1)
So, Meta.parse will convert an appropriate string to an expression. And if you eval the result, the expression will be evaluated. Note that printing a simple expression removes the outer :( )
julia> expr = Meta.parse("1 + 1")
:(1 + 1)
julia> print(expr)
1 + 1
julia> result = eval(expr)
2
Usually, macros are used to manipulate things before the usual evaluation of expressions; they are syntax transformations, mostly. Macros are performed before other source code is compiled/evaluated/executed.
Rather than seeking a macro that evaluates a string as if it were typed directly into the REPL (without quotes), use this function instead.
evalstr(x::AbstractString) = eval(Meta.parse(x))
While I do not recommend this next macro, it is good to know the technique.
A macro named <name>_str is used like this <name>"<string contents>" :
julia> macro eval_str(x)
:(eval(Meta.parse($x)))
end
julia> eval"1 + 1"
2
(p.s. do not reuse Base function names as variable names, use str not string)
Please let me know if there is something I have not addressed.
I am beginning my journey of learning Rust. I came across this line in Rust by Example:
However, unlike macros in C and other languages, Rust macros are expanded into abstract syntax trees, rather than string preprocessing, so you don't get unexpected precedence bugs.
Why is an abstract syntax tree better than string preprocessing?
If you have this in C:
#define X(A,B) A+B
int r = X(1,2) * 3;
The value of r will be 7, because the preprocessor expands it to 1+2 * 3, which is 1+(2*3).
In Rust, you would have:
macro_rules! X { ($a:expr,$b:expr) => { $a+$b } }
let r = X!(1,2) * 3;
This will evaluate to 9, because the compiler will interpret the expansion as (1+2)*3. This is because the compiler knows that the result of the macro is supposed to be a complete, self-contained expression.
That said, the C macro could also be defined like so:
#define X(A,B) ((A)+(B))
This would avoid any non-obvious evaluation problems, including the arguments themselves being reinterpreted due to context. However, when you're using a macro, you can never be sure whether or not the macro has correctly accounted for every possible way it could be used, so it's hard to tell what any given macro expansion will do.
By using AST nodes instead of text, Rust ensures this ambiguity can't happen.
A classic example using the C preprocessor is
#define MUL(a, b) a * b
// ...
int res = MUL(x + y, 5);
The use of the macro will expand to
int res = x + y * 5;
which is very far from the expected
int res = (x + y) * 5;
This happens because the C preprocessor really just does simple text-based substitutions, it's not really an integral part of the language itself. Preprocessing and parsing are two separate steps.
If the preprocessor instead parsed the macro like the rest of the compiler, which happens for languages where macros are part of the actual language syntax, this is no longer a problem as things like precedence (as mentioned) and associativity are taken into account.
I have an alias which I want to use in a macro:
foo = ....;
define my_macro (z) returns y {
$y = join $z in id, foo on id;
};
a = my_macro(b);
Alas, I get the error:
Undefined alias: macro_my_macro_foo_0
I can, of course, pass foo as en argument:
define my_macro (foo, z) returns y {
$y = join $z in id, $foo on id;
};
a = my_macro(foo,b);
Is this the right way?
If foo is actually a relatively complicated object, will it be recalculated for each macroexpansion of my_macro?
Yes the second approach is right one, you need to pass the alias as an argument to macro otherwise it will not be visible inside macro.
on the other side, alias defined inside the macro will not be access outside, in-case if you want to access the alias then use this format macro_<my macro_name>_<alias name suffixed with an instance>
I have simulated both the options
1. accessing alias from outside to inside macro(using argument)
2. accessing alias from inside macro to outside (using macro expanded name format)
example
in.txt
a,10,1000
b,20,2000
c,30,3000
in1.txt
10,aaa
20,bbb
30,ccc
Pigscript:
define my_macro (foo,z) returns y {
$y = join $z by g1, $foo by f2;
test = FOREACH $y generate $0,$2;
};
foo = LOAD 'in.txt' USING PigStorage(',') AS (f1,f2,f3);
b = LOAD 'in1.txt' USING PigStorage(',') AS (g1,g2);
C = my_macro(foo,b);
DUMP C;
--DUMP macro_my_macro_test_0;
Output of option1:
DUMP C
(10,aaa,a,10,1000)
(20,bbb,b,20,2000)
(30,ccc,c,30,3000)
Output of option2:
DUMP macro_my_macro_test_0
(10,a)
(20,b)
(30,c)
There are some restrictions in using the macro, like
1. not allowed inside nested for each stmt
2. not allowed to use any grunt commands
3. not allowed to include a user-defined schema
I suggest you to refer the below document link, this will definitely give some better ideas about macros and also how to use inside pig script.
http://pig.apache.org/docs/r0.13.0/cont.html#macros
If you run this code, the variable f seems to shadow the function f. Is there anyway to reach the function f?
func f (a:Int)->Int{
return a + 43
}
var f = {(a:Int) in a + 42}
var z = f(1)
println(z)
No.
In Swift, function declarations are simply shortcuts for what you did with that closure + variable thing. That is, function names are essentially constants and should always be viewed as such (you can even pass around the function name, without brackets, as a reference).
What you're doing is you're redeclaring the name f to the variable closure. It seems Swift has a compiler issue not complaining about this. However, this problem would never occur in good code, so it's not a real problem.
It can be a bit confusing, though.