Using unquote_splicing in macros with modified lists - macros

Elixir's unquote_splicing works without issues when directly unquoting passed lists. For example, calling the macro below Test1.wrap([1,2,3]) will correctly return [0,0,0,1,2,3,0,0,0].
defmodule Test1 do
defmacro wrap(nums) do
quote do
[0,0,0, unquote_splicing(nums), 0,0,0]
end
end
end
But if I make any changes to the list and then try calling unquote_splicing, Elixir won't even let me define the macro:
defmodule Test2 do
defmacro double_wrap(nums) do
quote do
doubles = Enum.map(unquote(nums), & &1*2)
[0,0,0, unquote_splicing(doubles), 0,0,0]
end
end
end
This will directly raise a compile error:
warning: variable "doubles" does not exist and is being expanded to "doubles()", please use parentheses to remove the ambiguity or change the variable name
iex:37: Test.double_wrap/1
** (CompileError) iex:37: undefined function doubles/0
(elixir) src/elixir_locals.erl:108: :elixir_locals."-ensure_no_undefined_local/3-lc$^0/1-0-"/2
(elixir) src/elixir_locals.erl:108: anonymous fn/3 in :elixir_locals.ensure_no_undefined_local/3
I have tried a bunch of things so far, such as:
Using nested quotes
Using bind_quoted
Going through Macro and Code docs
but nothing has worked and I can't figure out what I'm doing wrong.

What is returned by a macro, is directly injected in place of the calling code. Kernel.SpecialForms.unquote/1 (as well as unquote_splicing/1) is used to get access to the caller context. That is why your code raises: there is no local variable doubles defined in the caller context.
What you can do, would be to declare doubles outside of the quote block.
defmodule D do
defmacro double_wrap(nums) do
doubles = Enum.map(nums, & &1*2)
quote do
[0,0,0, unquote_splicing(doubles), 0,0,0]
end
end
end
require D
D.double_wrap [1,2,3]
#⇒ [0, 0, 0, 2, 4, 6, 0, 0, 0]
That said, this is happily resolved:
doubles = [1,2,3]
quote do: [0,0,0, unquote_splicing(doubles), 0,0,0]
#⇒ [0, 0, 0, 1, 2, 3, 0, 0, 0]
And this is not, because there is no doubles in the caller context:
quote do
doubles = [1,2,3]
[0,0,0, unquote_splicing(doubles), 0,0,0]
end
#⇒ ☠️ ** (CompileError) iex:7: undefined function doubles/0
The error message says undefined function, because elixir tries a local variable, and if it does not find it in the current context, it attempts to call the function with this name and the arity zero.

There's no need to reach outside the quote block to retrieve the value of doubles when doubles is defined inside the quote block. Variables that are defined inside a quote block automatically have their values embedded in the AST. Therefore, you can use the function List.flatten():
defmodule A do
defmacro double_wrap(nums) do
quote do
doubles = Enum.map(unquote(nums), & &1*2)
List.flatten [0,0,0, doubles, 0,0,0]
end
end
end
In iex:
~/elixir_programs$ iex
Erlang/OTP 20 [erts-9.2] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:10] [hipe] [kernel-poll:false]
Interactive Elixir (1.8.2) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> c "a.ex"
[A]
iex(2)> require A
A
iex(3)> A.double_wrap [1, 2, 3]
[0, 0, 0, 2, 4, 6, 0, 0, 0]
iex(4)>

Related

Calling macro from within generated function in Julia

I have been messing around with generated functions in Julia, and have come to a weird problem I do not understand fully: My final goal would involve calling a macro (more specifically #tullio) from within a generated function (to perform some tensor contractions that depend on the input tensors). But I have been having problems, which I narrowed down to calling the macro from within the generated function.
To illustrate the problem, let's consider a very simple example that also fails:
macro my_add(a,b)
return :($a + $b)
end
function add_one_expr(x::T) where T
y = one(T)
return :( #my_add($x,$y) )
end
#generated function add_one_gen(x::T) where T
y = one(T)
return :( #my_add($x,$y) )
end
With these declarations, I find that eval(add_one_expr(2.0)) works just as expected and returns and expression
:(#my_add 2.0 1.0)
which correctly evaluates to 3.0.
However evaluating add_one_gen(2.0) returns the following error:
MethodError: no method matching +(::Type{Float64}, ::Float64)
Doing some research, I have found that #generated actually produces two codes, and in one only the types of the variables can be used. I think this is what is happening here, but I do not understand what is happening at all. It must be some weird interaction between macros and generated functions.
Can someone explain and/or propose a solution? Thank you!
I find it helpful to think of generated functions as having two components: the body and any generated code (the stuff inside a quote..end). The body is evaluated at compile time, and doesn't "know" the values, only the types. So for a generated function taking x::T as an argument, any references to x in the body will actually point to the type T. This can be very confusing. To make things clearer, I recommend the body only refer to types, never to values.
Here's a little example:
julia> #generated function show_val_and_type(x::T) where {T}
quote
println("x is ", x)
println("\$x is ", $x)
println("T is ", T)
println("\$T is ", $T)
end
end
show_val_and_type
julia> show_val_and_type(3)
x is 3
$x is Int64
T is Int64
$T is Int64
The interpolated $x means "take the x from the body (which refers to T) and splice it in.
If you follow the approach of never referring to values in the body, you can test generated functions by removing the #generated, like this:
julia> function add_one_gen(x::T) where T
y = one(T)
quote
#my_add(x,$y)
end
end
add_one_gen
julia> add_one_gen(3)
quote
#= REPL[42]:4 =#
#= REPL[42]:4 =# #my_add x 1
end
That looks reasonable, but when we test it we get
julia> add_one_gen(3)
ERROR: UndefVarError: x not defined
Stacktrace:
[1] macro expansion
# ./REPL[48]:4 [inlined]
[2] add_one_gen(x::Int64)
# Main ./REPL[48]:1
[3] top-level scope
# REPL[49]:1
So let's see what the macro gives us
julia> #macroexpand #my_add x 1
:(Main.x + 1)
It's pointing to Main.x, which doesn't exist. The macro is being too eager, and we need to delay its evaluation. The standard way to do this is with esc. So finally, this works:
julia> macro my_add(a,b)
return :($(esc(a)) + $(esc(b)))
end
#my_add
julia> #generated function add_one_gen(x::T) where T
y = one(T)
quote
#my_add(x,$y)
end
end
add_one_gen
julia> add_one_gen(3)
4

Can I expand a macro inside a quote of another macro in Elixir?

Given the below code, the line use Composite, user_opts: user_opts ends up as [{:user_opts, [line: 3, counter: {MockUserNode1, 2}], Automaton.Node}] inside the using(opts) as opts. Is there any way to inject that code inside the outer macro?
defmacro __using__(user_opts) do
a =
if Enum.member?(Composite.types(), user_opts[:node_type]) do
IO.inspect(user_opts)
quote bind_quoted: [user_opts: user_opts] do
use DynamicSupervisor
use Composite, user_opts: user_opts
end
else
quote do: use(Action)
end
end
Answering the question stated: it’s perfectly possible to call macros from inside other macros, the just inject the AST recursively in the end.
defmodule DeeplyUsed do
defmacro __using__(opts) do
quote bind_quoted: [opts: opts] do
opts
end
end
end
defmodule Used do
defmacro __using__(opts) do
quote bind_quoted: [opts: opts] do
use DeeplyUsed, opts: opts
end
end
end
defmodule Using do
use Used, line: 3, counter: {MockUserNode1, 2}
end
That said, your issue is induced.
Sidenote: [{:user_opts, _, Automaton.Node}] looks indeed very suspicious there, that’s not how keyword lists are being quoted. Start with explicit unquoting and logging what comes to user_opts there.
defmacro __using__(user_opts) do
IO.inspect(user_opts, label: "Outside")
quote do
IO.inspect(unquote(user_opts), label: "Inside")
use Composite, user_opts: unquote(user_opts)
end
end

Macros vs functions that accept expressions

When I try the following code snippet, I got that the variable i is not found. Why is that ?
function evalMyExpr(expr,n)
for i in 1:n
eval(expr)
end
end
expr1 = Meta.parse("println(\"hello\")")
expr2 = Meta.parse("println(string(i))")
evalMyExpr(expr1,2) # ok
evalMyExpr(expr2,2) # UndefVarError: i not defined
Note that if I transform it in a macro it works:
macro evalMyExprMacro(expr,n)
quote
for i in 1:$n
$expr
end
end
end
#evalMyExprMacro println(string(i)) 2 # ok
More in general, which is the difference between a function that accepts expressions as parameters and a macro?
Expressions passed to functions are just normal values that are processed at run time. The reason why code fails when it is passed expr2 is because eval evaluates expressions in global scope (in general it is not recommended to use eval in functions). Therefore, as probably variable i is not defined in global scope in your case, you get an error. See an example when i is defined in global scope:
julia> i = 1000
1000
julia> function evalMyExpr(expr,n)
for i in 1:n
eval(expr)
end
end
evalMyExpr (generic function with 1 method)
julia> expr2 = Meta.parse("println(string(i))")
:(println(string(i)))
julia>
julia> evalMyExpr(expr2,2)
1000
1000
Now - in marcos expressions are processed in compile time (before the code is run) so the expression you use is injected into the code generated by the macro that is executed afterwards. You can see the effect by using #macroexpand:
julia> macro evalMyExprMacro(expr,n)
quote
for i in 1:$n
$expr
end
end
end
#evalMyExprMacro (macro with 1 method)
julia> #macroexpand #evalMyExprMacro println(string(i)) 2
quote
#= REPL[23]:3 =#
for #6#i = 1:2
#= REPL[23]:4 =#
(Main.println)((Main.string)(#6#i))
end
end
Observe that the variable name was changed by the macro processing mechanizm to #6#i and it matches the name of the variable that is used in the for loop.

Passing computed list to an Elixir macro

I have a map that I want to use a single source of truth for a couple of functions. Let's say it is:
source_of_truth = %{a: 10, b: 20}
I'd like the keys of that map to be values of EctoEnum. EctoEnum provides a macro defenum that I should use like this:
defenum(
EnumModule,
:enum_name,
[:a, :b]
)
I don't want to repeat [:a, :b] part. I'd like to use the keys from the map instead like this:
defenum(
EnumModule,
:enum_name,
Map.keys(source_of_truth)
)
It doesn't work because defenum macro expects a plain list.
I thought I could do it by defining my own macro like this:
defmacro dynamic_enum(enum_module, enum_name, enum_values) do
quote do
defenum(
unquote(enum_module),
unquote(enum_name),
unquote(enum_values)
)
end
end
and then call:
dynamic_enum(EnumModule, :enum_name, Map.keys(source_of_truth))
However, it doest the same thing: enum_values is not a precomputed list but AST for Map.get. My next approach was:
defmacro dynamic_enum(enum_module, enum_name, enum_values) do
quote do
values = unquote(enum_values)
defenum(
unquote(enum_module),
unquote(enum_name),
?
)
end
end
Not sure what I could put where the ? is. I can't just put values because it is a variable and not a list. I can't put unquote(values) either.
A solution that sort of works is this one:
defmacro dynamic_enum(enum_module, enum_name, enum_values) do
{values, _} = Code.eval_quoted(enum_values)
quote do
defenum(
unquote(enum_module),
unquote(enum_name),
unquote(values)
)
end
end
However, the docs say that using eval_quoted inside a macro is a bad practice.
[EDIT]
A solution with Macro.expand does not work either because it doesn't actually evaluate anything. The expansion stops at:
Expanded: {{:., [],
[
{:__aliases__, [alias: false, counter: -576460752303357631], [:Module]},
:get_attribute
]}, [],
[
{:__MODULE__, [counter: -576460752303357631], Kernel},
:keys,
[
{:{}, [],
[
TestModule,
:__MODULE__,
0,
[
file: '...',
line: 16
]
]}
]
]}
So it does not expand to the list as we expected.
[\EDIT]
What is a good solution for that problem?
As stated in the documentation for Macro.expand/2
The following contents are expanded:
Macros (local or remote)
Aliases are expanded (if possible) and return atoms
Compilation environment macros (__CALLER__/0, __DIR__/0, __ENV__/0 and __MODULE__/0)
Module attributes reader (#foo)
Emphasis is mine. So the possibility would be to use module attributes with Macro.expand/2.
defmacro dynamic_enum(enum_module, enum_name, enum_values) do
IO.inspect(enum_values, label: "Passed")
expanded = Macro.expand(enum_values, __CALLER__)
IO.inspect(expanded, label: "Expanded")
quote do
defenum(
unquote(enum_module),
unquote(enum_name),
unquote(expanded)
)
end
end
And call it like:
#source_of_truth %{a: 10, b: 20}
#keys Map.keys(#source_of_truth)
def test_attr do
dynamic_enum(EnumModuleA, :enum_name_a, #keys)
end
FWIW, the full code:
$ \cat lib/eenum.ex
defmodule Eenum do
import EctoEnum
defmacro dynamic_enum(enum_module, enum_name, enum_values) do
IO.inspect(enum_values, label: "Passed")
expanded = Macro.expand(enum_values, __CALLER__)
IO.inspect(expanded, label: "Expanded")
quote do
defenum(
unquote(enum_module),
unquote(enum_name),
unquote(expanded)
)
end
end
end
$ \cat lib/tester.ex
defmodule Tester do
import Eenum
#source_of_truth %{a: 10, b: 20}
#keys Map.keys(#source_of_truth)
def test_attr do
dynamic_enum(EnumModuleA, :enum_name_a, #keys)
end
end
FWIW 2. To be able to call dynamic_enum as shown above from the module scope, all you need is (surprise :) another module scope, already compiled at the moment of macro invocation:
defmodule Defs do
#source_of_truth %{a: 10, b: 20}
#keys Map.keys(#source_of_truth)
defmacro keys, do: Macro.expand(#keys, __CALLER__)
end
defmodule Tester do
import Defs
import Eenum
dynamic_enum(EnumModuleA, :enum_name_a, keys())
end
FWIW 3. The latter (explicit module with definitions) will work even without a necessity to have module attributes:
defmodule Defs do
defmacro keys, do: Macro.expand(Map.keys(%{a: 10, b: 20}), __CALLER__)
end
defmodule Tester do
import Defs
import Eenum
dynamic_enum(EnumModuleA, :enum_name_a, keys())
end
The rule of thumb is when you find yourself in a need to invoke Code.eval_quoted/3, put this code into the independent module and make compiler invoke this code compilation for you. For functions is works on the module level, for module level it should be put into another module to make module context (aka __CALLER__ and __ENV__) available.
I battled with the same problem a while back. Basically you can build your syntax tree in a quote, using unquote to inject your dynamic value, and then use Code.eval_quoted to eval the macros:
options = Map.keys(source_of_truth)
Code.eval_quoted(
quote do
EctoEnum.defenum(MyEnum, :type_name, unquote(options))
end,
[],
__ENV__
)

SAS IML use of Mattrib with Macro (symget) in a loop

In an IML proc I have several martices and several vectors with the names of columns:
proc IML;
mydata1 = {1 2 3, 2 3 4};
mydata2 = {1 2, 2 3};
names1 = {'red' 'green' 'blue'};
names2 = {'black' 'white'};
To assign column names to columns in matrices one can copypaste the mattrib statement enough times:
/* mattrib mydata1 colname=names1;*/
/* mattrib mydata2 colname=names2;*/
However, in my case the number of matrices is defined at execution, thus a do loop is needed. The following code
varNumb=2;
do idx=1 to varNumb;
call symputx ('mydataX', cat('mydata',idx));
call symputx ('namesX', cat('names',idx));
mattrib (symget('mydataX')) colname=(symget('namesX'));
end;
print (mydata1[,'red']) (mydata2[,'white']);
quit;
however produces the "Expecting a name" error on the first symget.
Similar question Loop over names in SAS-IML? offers the macro workaround with symget, what produces an error here.
What is the correct way of using mattrib with symget? Is there other way of making a variable from a string than macro?
Any help would be appreciated.
Thanks,
Alex
EDIT1
The problem is in the symget function. The &-sign resolves the name of the matrix contained in the macro variable, the symget only returns the name of the macro.
proc IML;
mydata1 = {1 2 3};
call symputx ('mydataX', 'mydata1');
mydataNew = (symget('mydataX'));
print (&mydataX);
print (symget("mydataX"));
print mydataNew;
quit;
results in
mydata1 :
1 2 3
mydata1
mydataNew :
mydata1
Any ideas?
EDIT2
Function value solves the symget problem in EDIT1
mydataNew = value(symget('mydataX'));
print (&mydataX);
print (value(symget("mydataX")));
print mydataNew;
The mattrib issue but persists.
SOLVED
Thanks Rick, you have opened my eyes to CALL EXECUTE() statement.
When you use CALL SYMPUTX, you should not use quotes for the second argument. Your statement
call symputx ('mydataX', 'mydata1');
assigns the string 'mydata1' to the macro variable.
In general, trying to use macro variables in SAS/IML loops often results in complicated code. See the article Macros and loops in the SAS/IML language for an indication of the issues caused by trying to combine a macro preprocessor with an interactive language. Because the MATTRIB statement expects a literal value for the matrix name, I recomend that you use CALL EXECUTE rather than macro substitution to execute the MATTRIB statement.
You are also having problems because a macro variable is always a scalar string, whereas the column name is a vector of strings. Use the ROWCAT function to concatenate the vector of names into a single string.
The following statements accomplish your objective without using macro variables:
/* Use CALL EXECUTE to set matrix attributes dynamically.
Requires that matrixName and varNames be defined at main scope */
start SetMattrib;
cmd = "mattrib " + matrixName + " colname={" + varNames + "};";
*print cmd; /* for debugging */
call execute(cmd);
finish;
varNumb=2;
do idx=1 to varNumb;
matrixName = cat('mydata',idx);
varNames = rowcat( value(cat('names',idx)) + " " );
run SetMattrib;
end;