I have been messing around with generated functions in Julia, and have come to a weird problem I do not understand fully: My final goal would involve calling a macro (more specifically #tullio) from within a generated function (to perform some tensor contractions that depend on the input tensors). But I have been having problems, which I narrowed down to calling the macro from within the generated function.
To illustrate the problem, let's consider a very simple example that also fails:
macro my_add(a,b)
return :($a + $b)
end
function add_one_expr(x::T) where T
y = one(T)
return :( #my_add($x,$y) )
end
#generated function add_one_gen(x::T) where T
y = one(T)
return :( #my_add($x,$y) )
end
With these declarations, I find that eval(add_one_expr(2.0)) works just as expected and returns and expression
:(#my_add 2.0 1.0)
which correctly evaluates to 3.0.
However evaluating add_one_gen(2.0) returns the following error:
MethodError: no method matching +(::Type{Float64}, ::Float64)
Doing some research, I have found that #generated actually produces two codes, and in one only the types of the variables can be used. I think this is what is happening here, but I do not understand what is happening at all. It must be some weird interaction between macros and generated functions.
Can someone explain and/or propose a solution? Thank you!
I find it helpful to think of generated functions as having two components: the body and any generated code (the stuff inside a quote..end). The body is evaluated at compile time, and doesn't "know" the values, only the types. So for a generated function taking x::T as an argument, any references to x in the body will actually point to the type T. This can be very confusing. To make things clearer, I recommend the body only refer to types, never to values.
Here's a little example:
julia> #generated function show_val_and_type(x::T) where {T}
quote
println("x is ", x)
println("\$x is ", $x)
println("T is ", T)
println("\$T is ", $T)
end
end
show_val_and_type
julia> show_val_and_type(3)
x is 3
$x is Int64
T is Int64
$T is Int64
The interpolated $x means "take the x from the body (which refers to T) and splice it in.
If you follow the approach of never referring to values in the body, you can test generated functions by removing the #generated, like this:
julia> function add_one_gen(x::T) where T
y = one(T)
quote
#my_add(x,$y)
end
end
add_one_gen
julia> add_one_gen(3)
quote
#= REPL[42]:4 =#
#= REPL[42]:4 =# #my_add x 1
end
That looks reasonable, but when we test it we get
julia> add_one_gen(3)
ERROR: UndefVarError: x not defined
Stacktrace:
[1] macro expansion
# ./REPL[48]:4 [inlined]
[2] add_one_gen(x::Int64)
# Main ./REPL[48]:1
[3] top-level scope
# REPL[49]:1
So let's see what the macro gives us
julia> #macroexpand #my_add x 1
:(Main.x + 1)
It's pointing to Main.x, which doesn't exist. The macro is being too eager, and we need to delay its evaluation. The standard way to do this is with esc. So finally, this works:
julia> macro my_add(a,b)
return :($(esc(a)) + $(esc(b)))
end
#my_add
julia> #generated function add_one_gen(x::T) where T
y = one(T)
quote
#my_add(x,$y)
end
end
add_one_gen
julia> add_one_gen(3)
4
I have a procedure
f:=proc(x,option1,option2) ... end proc;
In my case option1 is always an integer and option2 is either a list or something else (including integer). Both options are optional, so these commands work as expected:
f(x);
f(x,3);
f(x,4,[1,2]);
f(x,5,3);
f(x,6,expand);
But if option1 isn't specified then I don't know an easy way to deal with it since Maple doesn't allow the usage like
f(x,,3);
f(x,,[1,2]);
I can make it understand
f(x,[1,2]);
but I still have a problem with
f(x,3);
since it's not clear if 3 is option1 or option2. I can rewrite the code to understand function calls in this format
f(x,[1,option1],[2,option2]);
f(x,[1,option1]);
f(x,[2,option2]);
but I'm curious if there is a simpler way to achieve that since for some Maple functions (like plot) the order of most options doesn't matter.
As others already mentioned one solution is using "kwarg" (keyword argument). You can also use _passed or _rest. You can read more in Maple help pages or Maple programming guide (https://www.maplesoft.com/documentation_center/Maple2021/ProgrammingGuide.pdf).
Here is just an example how you can use them. _passed is for when you want to say whatever that has been passed to your procedure, and _rest is for whatever that has been passed to your procedure except the ones that are already assigned to the parameters you mentioned inside the parentheses in front of proc. Let's say we want to define a procedure with 1 necessary argument and possible 2 optional arguments. If there are two optional arguments given, we assume the first one is always option1 and the second one is option2, but if only one optional argument is given, then depending on if it is of type integer or not it will be option1 or option2 respectively.
To ask the number of passed or the rest of passed arguments you can use _npassed and _nrest. And the command assigned() checks if something is assigned a value or not. You can check if something is of a specific type, by type(-,-) or - :: -. So here is the simple code.
test := proc( x )
local option1, option2:
if _nrest = 2 then
option1 := _rest[1]:
option2 := _rest[2]:
elif _nrest = 1 then
if _rest[1] :: integer then
option1 := _rest[1]:
else
option2 := _rest[1]:
end if:
end if:
printf( "necessary argument is %a\n", x ):
if assigned( option1 ) then
printf( "option 1 is given and is %d\n", option1 ):
end if:
if assigned( option2 ) then
printf( "option 2 is given and is %a\n", option2 ):
end if:
end proc:
Here is a screenshot of the output of the above procedure for different inputs.
Most of the plotting commands use Maple's more modern argument-processing to manage procedure options.
In particular most options to plotting commands are provided as so-called keyword options. That automatically provides the functionlity in which the location (of such options) doesn't matter.
For example,
f:=proc(v,
{ord::{integer,NoUserValue}:=':-NoUserValue'},
{special::{integer,list,NoUserValue}:=':-NoUserValue'});
print(':-ord'=ord, ':-special'=special);
end proc:
f(x);
ord = NoUserValue, special = NoUserValue
f(x,ord=3);
ord = 3, special = NoUserValue
f(x,special=5);
ord = NoUserValue, special = 5
f(x,special=5,ord=3);
ord = 3, special = 5
f(x,ord=3,special=5);
ord = 3, special = 5
As you've noticed, you [logically] cannot use multiple *positional/ordered" parameters if both have the same type and some earlier ones are missing.
If you really wanted you could make one of those options into a positional parameter, although naturally that would lose its flexibility of arbitrary placement. For example,
restart;
f2:=proc(v,
ord::{integer,NoUserValue}:=':-NoUserValue',
{special::{integer,list,NoUserValue}:=':-NoUserValue'});
print(':-ord'=ord, ':-special'=special);
end proc:
f2(x);
f2(x,3);
f2(x,special=5);
f2(x,special=5,3);
f2(x,3,special=5);
restart;
f3:=proc(v,
special::{integer,list,NoUserValue}:=':-NoUserValue',
{ord::{integer,NoUserValue}:=':-NoUserValue'});
print(':-ord'=ord, ':-special'=special);
end proc:
f3(x);
f3(x,5);
f3(x,ord=3);
f3(x,ord=3,5);
f3(x,5,ord=3);
There are too many variants to show them all here, sensibly.
You don't have to use the name "NoUserValue" as the default values.
Use keyword arguments.
f:=proc(x,{op1::integer:=0,op2::{list,integer}:={}},$)
if has([_passed],'op1') then
print("op1 =",op1);
else
print("op1 not passed");
fi;
if has([_passed],'op2') then
print("op2 =",op2);
else
print("op2 not passed");
fi;
#rest of code
end proc;
Now you can do
f(x,'op2'=[1,2,3])
"op1 not passed"
"op2 =", [1, 2, 3]
And
f(x,'op1'=99)
"op1 =", 99
"op2 not passed"
And
f(x)
"op1 not passed"
"op2 not passed"
And
f(x,'op1'=99,'op2'=[1,2,3])
"op1 =", 99
"op2 =", [1, 2, 3]
And
f(x,'op1'=99,'op2'=19827)
"op1 =", 99
"op2 =", 19827
Make sure to use 'op1'=value when calling, and not op1=value
I was following along this notebook (originally written in Julia 0.x, I am using Julia 1.7.1). One of the cells defines the following macro.
macro twice(ex)
quote
$ex
$ex
end
end
On the very next cell, this macro is invoked.
x = 0
#twice println(x += 1)
Replicating this in the REPL (for brevity) results in the following error.
ERROR: syntax: invalid assignment location "Main.x" around REPL[1]:3
Stacktrace:
[1] top-level scope
# REPL[4]:1
So, I understand that x += 1 is somehow causing this problem, but after going through the docs (metaprogramming), I could not figure out why exactly this is an invalid assignment or how to fix this.
#macroexpand #twice println(x += 1) successfully returns the following.
quote
#= REPL[1]:3 =#
Main.println(Main.x += 1)
#= REPL[1]:4 =#
Main.println(Main.x += 1)
end
So, I tried evaling this in the top-level without the Main.s, and it evaluates successfully.
x = 0
eval(quote
println(x += 1)
println(x += 1)
end)
Output:
1
2
But, if I add the module name explicitly, it throws a different error.
eval(quote
Main.println(Main.x += 1)
Main.println(Main.x += 1)
end)
ERROR: cannot assign variables in other modules
Stacktrace:
[1] setproperty!(x::Module, f::Symbol, v::Int64)
# Base ./Base.jl:36
[2] top-level scope
# REPL[14]:3
[3] eval
# ./boot.jl:373 [inlined]
[4] eval(x::Expr)
# Base.MainInclude ./client.jl:453
[5] top-level scope
# REPL[14]:1
I tried a few other things, but these are the only things that I think might be getting somewhere.
Why exactly is the assignment in the first macro code block invalid? Is it for the same reason that evaling in the top-level with the module Main specified fails?
How can this invalid assignment be circumvented, OR how do I port this to Julia 1.7?
Use esc (This "prevents the macro hygiene pass from turning embedded variables into gensym variables"):
julia> macro twice(ex)
esc(quote
$ex
$ex
end)
end;
julia> x=1
1
julia> #twice println(x += 1)
2
3
I am a noob at metaprogramming so maybe I am not understanding this. I thought the purpose of the #nloops macro in Base.Cartesian was to make it possible to code an arbitrary number of nested for loops, in circumstances where the dimension is unknown a priori. In the documentation for the module, the following example is given:
#nloops 3 i A begin
s += #nref 3 A i
end
which evaluates to
for i_3 = 1:size(A,3)
for i_2 = 1:size(A,2)
for i_1 = 1:size(A,1)
s += A[i_1,i_2,i_3]
end
end
end
Here, the number 3 is known a priori. For my purposes, however, and for the purposes that I thought nloops was created, the number of nested levels is not known ahead of time. So I would not be able to hard code the integer 3. Even in the documentation, it is stated:
The (basic) syntax of #nloops is as follows:
The first argument must be an integer (not a variable) specifying the number of loops.
...
If I assign an integer value - say the dimension of an array that is passed to a function - to some variable, the nloops macro no longer works:
b = 3
#nloops b i A begin
s += #nref b A i
end
This returns an error:
ERROR: LoadError: MethodError: no method matching _nloops(::Symbol, ::Symbol, ::Symbol, ::Expr)
Closest candidates are:
_nloops(::Int64, ::Symbol, ::Symbol, ::Expr...) at cartesian.jl:43
...
I don't know how to have nloops evaluate the b variable as an integer rather than a symbol. I have looked at the documentation and tried various iterations of eval and other functions and macros but it is either interpreted as a symbol or an Expr. What is the correct, julian way to write this?
See supplying the number of expressions:
julia> A = rand(4, 4, 3) # 3D array (Array{Int, 3})
A generated function is kinda like a macro, in that the resulting expression is not returned, but compiled and executed on invocation/call, it also sees the type (and their type parameters of course) of the arguments, ie:
inside the generated function, A is Array{T, N}, not the value of the array.
so T is Int and N is 3!
Here inside the quoted expression, N is interpolated into the expression, with the syntax $N, which evaluates to 3:
julia> #generated function mysum(A::Array{T,N}) where {T,N}
quote
s = zero(T)
#nloops $N i A begin
s += #nref $N A i
end
s
end
end
mysum (generic function with 1 method)
julia> mysum(A)
23.2791638775186
You could construct the expression and then evaluate it, ie.:
julia> s = 0; n = 3;
julia> _3loops = quote
#nloops $n i A begin
global s += #nref $n A i
end
end
quote
#nloops 3 i A begin
global s += #nref(3, A, i)
end
end
julia> eval(_3loops)
julia> s
23.2791638775186
I have scrubbed manually the LineNumberNodes from the AST for readability (there is also MacroTools.prettify, that does it for you).
Running this example in the REPL needs to declare s as global inside the loop in Julia 1.0.
This question builds off of a previous SO question which was for building expressions from expressions inside of of a macro. However, things got a little trucker when quoting the whole expression. For example, I want to build the expression :(name=val). The following:
macro quotetest(name,val)
quote
nm = Meta.quot($(QuoteNode(name)))
v = Meta.quot($(QuoteNode(val)))
println(nm); println(typeof(nm))
println(v); println(typeof(val))
end
end
#quotetest x 5 # Test case: build :(x=5)
prints out
:x
Expr
$(Expr(:quote, 5))
Expr
showing that I am on the right path: nm and val are the expressions that I want inside of the quote. However, I can't seem to apply the previous solution at this point. For example,
macro quotetest(name,val)
quote
nm = Meta.quot($(QuoteNode(name)))
v = Meta.quot($(QuoteNode(val)))
println(nm); println(typeof(nm))
println(v); println(typeof(v))
println(:($(Expr(:(=),$(QuoteNode(nm)),$(QuoteNode(val))))))
end
end
fails, saying nm is not defined. I tried just interpolating without the QuoteNode, escaping the interpolation $(esc(nm)), etc. I can't seem to find out how to make it build the expression.
I think you are using $ signs more than you need to. Is this what you're looking for?
julia> macro quotetest(name,val)
quote
expr = :($$(QuoteNode(name)) = $$(QuoteNode(val)))
println(expr)
display(expr)
println(typeof(expr))
end
end
#quotetest (macro with 1 method)
julia> #quotetest test 1
test = 1
:(test = 1)
Expr