Defining Julia types using macros - macros

Let's say I want to define a series of structs that will be used as parametric types for some other struct later on. For instance, I would like to have something like
abstract type Letter end
struct A <: Letter end
struct B <: Letter end
...etc...
The idea I've had is to define a macro which takes a string of the name of the struct I want to create and defines it as well as some basic methods. I would then execute the macro in a loop for all names and get all my structs defined at compile time. Something like this:
const LETTERS = ["A","B","C","D"]
abstract type Letter end
macro _define_type(ex)
lines = Vector{Expr}()
push!(lines, Meta.parse("export $ex"))
push!(lines, Meta.parse("struct $ex <: Letter end"))
push!(lines, Meta.parse("string(::Type{$ex}) = \"$ex\""))
return Expr(:block,lines...)
end
#_define_type "A"
for ex in LETTERS
#_define_type ex
end
The first way of executing the macro (with a single string) works and does what I want. However, when I execute the macro in a loop it does not. It tells me some variables are declared both as local and global variables.
Can someone explain what is happening? I believe it may be solved by a proper use of esc, but I can't figure out how.
Thanks!
Edit: Thank you all for the amazing responses! Got my code running!

I think this is what you are trying to do:
module MyModule
abstract type Letter end
const LETTERS = ["A", "B", "C", "D"]
for letter in LETTERS
sym = Symbol(letter)
#eval begin
export $sym
struct $sym <: Letter end
Base.string(::Type{$sym}) = $letter
end
end
end
using .MyModule
string(A) # "A"
See the Code Generation section for more details:
https://docs.julialang.org/en/v1/manual/metaprogramming/#Code-Generation

Okay, the problem here is that in Julia for loops introduce a separate, local scope, while struct definitions need to be in the global scope. So your macro fails because it creates struct definitions in the local scope of the for loop.
A way to get around this is to use #eval, to ensure your struct definitions are evaluated in the global scope. In that case, you don't need to create a macro for that, just have a simple loop like this:
abstract type Letter end
const LETTERS = [:A, :B, :C, :D, :E]
for ex in LETTERS
#eval struct $ex <: Letters end
end
You can even put that loop in a function and it will still work. The defined structs can have fields, as #eval covers the entire code block that follows it.
Note that LETTERS must contain symbols rather than strings for this to work correctly. It's easy enough to convert a vector of strings into a vector of symbols using Symbol.(vector_of_strings).

While there are other ways of achieving what you want to do I believe the core issue is understanding the nature of macros. From the docs (emphasis mine):
Macros are necessary because they execute when code is parsed, therefore, macros allow the programmer to generate and include fragments of customized code before the full program is run.
So the macro in your loop does not "see" the values "A", "B", "C" and "D", it sees the expression: ex. To demonstrate this try using #macroexpand:
julia> #macroexpand #_define_type ex
quote
export ex
struct ex <: Main.Letter
#= none:1 =#
end
var"#11#string"(::Main.Type{Main.ex}) = begin
#= none:1 =#
"ex"
end
end
As you can see the actual value of the variable ex does not matter. With this in mind let's look at the actual error you get. You can reproduce it like this:
julia> for ex in ["A"]
struct ex <: Letter
end
end
ERROR: syntax: variable "ex" declared both local and global
Stacktrace:
[1] top-level scope
# REPL[52]:1
You can probably see that this is not what you want, but why this specific error? The reason is that structs are implicitly global while the loop variable is local.
Here is a possible solution that uses a macro that takes a variable number of arguments instead (I also switched to providing the expression directly instead of as a string):
abstract type Letter end
macro _define_types(exprs...)
blocks = map(exprs) do ex
name = string(ex)
quote
export $ex
struct $ex <: Letter end
Base.string(::Type{$ex}) = $name
end
end
Expr(:block, blocks...)
end
#_define_types A
#_define_types B C D

Related

Calling macro from within generated function in Julia

I have been messing around with generated functions in Julia, and have come to a weird problem I do not understand fully: My final goal would involve calling a macro (more specifically #tullio) from within a generated function (to perform some tensor contractions that depend on the input tensors). But I have been having problems, which I narrowed down to calling the macro from within the generated function.
To illustrate the problem, let's consider a very simple example that also fails:
macro my_add(a,b)
return :($a + $b)
end
function add_one_expr(x::T) where T
y = one(T)
return :( #my_add($x,$y) )
end
#generated function add_one_gen(x::T) where T
y = one(T)
return :( #my_add($x,$y) )
end
With these declarations, I find that eval(add_one_expr(2.0)) works just as expected and returns and expression
:(#my_add 2.0 1.0)
which correctly evaluates to 3.0.
However evaluating add_one_gen(2.0) returns the following error:
MethodError: no method matching +(::Type{Float64}, ::Float64)
Doing some research, I have found that #generated actually produces two codes, and in one only the types of the variables can be used. I think this is what is happening here, but I do not understand what is happening at all. It must be some weird interaction between macros and generated functions.
Can someone explain and/or propose a solution? Thank you!
I find it helpful to think of generated functions as having two components: the body and any generated code (the stuff inside a quote..end). The body is evaluated at compile time, and doesn't "know" the values, only the types. So for a generated function taking x::T as an argument, any references to x in the body will actually point to the type T. This can be very confusing. To make things clearer, I recommend the body only refer to types, never to values.
Here's a little example:
julia> #generated function show_val_and_type(x::T) where {T}
quote
println("x is ", x)
println("\$x is ", $x)
println("T is ", T)
println("\$T is ", $T)
end
end
show_val_and_type
julia> show_val_and_type(3)
x is 3
$x is Int64
T is Int64
$T is Int64
The interpolated $x means "take the x from the body (which refers to T) and splice it in.
If you follow the approach of never referring to values in the body, you can test generated functions by removing the #generated, like this:
julia> function add_one_gen(x::T) where T
y = one(T)
quote
#my_add(x,$y)
end
end
add_one_gen
julia> add_one_gen(3)
quote
#= REPL[42]:4 =#
#= REPL[42]:4 =# #my_add x 1
end
That looks reasonable, but when we test it we get
julia> add_one_gen(3)
ERROR: UndefVarError: x not defined
Stacktrace:
[1] macro expansion
# ./REPL[48]:4 [inlined]
[2] add_one_gen(x::Int64)
# Main ./REPL[48]:1
[3] top-level scope
# REPL[49]:1
So let's see what the macro gives us
julia> #macroexpand #my_add x 1
:(Main.x + 1)
It's pointing to Main.x, which doesn't exist. The macro is being too eager, and we need to delay its evaluation. The standard way to do this is with esc. So finally, this works:
julia> macro my_add(a,b)
return :($(esc(a)) + $(esc(b)))
end
#my_add
julia> #generated function add_one_gen(x::T) where T
y = one(T)
quote
#my_add(x,$y)
end
end
add_one_gen
julia> add_one_gen(3)
4

error: "struct" expression not at top level

function check(str,arg;type=DataType,max=nothing,min=nothing,description="")
#argcheck typeof(arg)==type
#argcheck arg>min
#argcheck arg<max
#argcheck typeof(description)==String
return arg
end
function constr(name,arg,field)
return :(function $name($arg,$field)
new(check($name,$arg,$field))
end)
end
macro creatStruct(name,arg)
code = Base.remove_linenums!(quote
struct $name
end
end)
print(arg)
append!(code.args[1].args[3].args,[constr(name,arg.args[1].args[1],arg.args[1].args[2])])
code
end
macro myStruct(name,arg)
#creatStruct name arg
end
#myStruct test12 (
(arg1,(max=10))
)
In my code above I'm trying to build a macro that Creates a struct, and within the struct, you can define an argument with boundaries (max, min) and description, etc.
I'm getting this error:
syntax: "#141#max = 10" is not a valid function argument name
and every time I'm trying to solve it, I get another error like:
LoadError: syntax: "struct" expression not at top level
So, I think my Code/Approach is not that cohesive. Anybody can suggest tips and/or another Approche.
You're attempting to make an argument name max with a default value of 10. The error is about max=10 not being a valid name (Symbol), while max is. The bigger issue is you're trying to put this in the struct expression instead of a constructor method:
struct Foo
bar::Float64
max::Int64
end
# constructor
Foo(bar, max=10) = Foo(bar, max)
So you have to figure out how to make an expression for a method with default values, too.
Your second error means that structs must be defined in the top-level. "Top-level" is like global scope but stricter in some contexts; I don't know the exact difference, but it definitely excludes local scopes (macro, function, etc). It looks like the issue is the expression returned by creatStruct being evaluated as code in myStruct, but the LoadError I'm getting has a different message. In any case, the error goes away if I make sure things stay as expressions:
macro myStruct(name,arg)
:(#creatStruct $name $arg)
end

How do I make a macro for a struct generate a function method matching on the struct?

Pardon any confused terminology in the title, but imagine I want to have a little macro to mark structs I create as usable for some nefarious purpose. I write this little module:
module Usable
export #usable, isusable
isusable(::Type{T}) where T = false
macro usable(expr::Expr)
name = expr.args[2]
return quote
$expr
Usable.isusable(::Type{$name}) = true # This in't working
end
end
end
However, trying to use my macro
julia> include("Usable.jl")
Main.Usable
julia> using Main.Usable
julia> #usable struct Foo
bar::String
end
results in
ERROR: UndefVarError: Foo not defined
The struct was evidently defined just fine
julia> Foo("soup")
Foo("soup")
so it would seem the definition is required earlier than I expect. I am obviously missing something, but I can't figure out what.
Always have a look at the output of your macros:
julia> #macroexpand #usable struct Foo
bar::String
end
quote
#= REPL[1]:11 =#
struct Foo
#= REPL[4]:2 =#
bar::Main.Usable.String
end
#= REPL[1]:13 =#
(Main.Usable.Usable).isusable(::Main.Usable.Type{Main.Usable.Foo}) = begin
#= REPL[1]:13 =#
true
end
end
The problem is that the macro output is expanded within the module it has been defined, which messes up what the names mean. In this case, we want Foo to refer to the namespace where it was defined, though:
quote
#= REPL[1]:11 =#
struct Foo
#= REPL[10]:2 =#
bar::String
end
#= REPL[1]:13 =#
Usable.isusable(::Type{Foo}) = begin
#= REPL[1]:13 =#
true
end
end
It's actually simple to get that -- just escape the output:
macro usable(expr::Expr)
name = expr.args[2]
return esc(quote
$expr
$Usable.isusable(::Type{$name}) = true
end)
end
But do read the macro documentation again. esc is quite intricate, and you don't want to blindly apply it to everything you write.
The other thing (and I hope to get this right) is to splice in the module itself -- $Usable -- instead of referring to it by name. Otherwise you might have a problem if you rename the module name outside.
In the described scenario almost always you should use Julia's powerful type system and multiple dispatch mechanism not a macro. (Perhaps you have a good reason to do so but this is information for others.)
The pattern is to simply define the desired behavior through an abstract type that is later inherited by a custom struct.
One example is here for adding a Comparable behavior to composite structs.
abstract type Comparable end
import Base.==
==(a::T, b::T) where T <: Comparable =
getfield.(Ref(a),fieldnames(T)) == getfield.(Ref(b),fieldnames(T))
And now using:
julia> struct MyStruct <: Comparable
f::Vector{String}
end;
julia> MyStruct(["hello"]) == MyStruct(["hello"])
true

Elixir Macros: Get unquoted parameter value outside of quote

I have a macro which gets a module name as parameter and I want to call a function on that module to get some data in order to generate the quote block.
Example:
defmacro my_macro(module) do
data = apply(module, :config, [])
# do something with data to generate the quote do end
end
Obviously, this doesn't work because the parameter value is quoted. I could fetch the data inside the quote block and act accordingly but that would put the whole logic inside the module that uses my macro which is quite dirty. I want to inject as little code as possible.
You can extract the module out by pattern matching with its quoted form: {:__aliases__, _, list} where list is a list of atoms which when concatenated with a dot (use Module.concat/1) produces the full module name.
defmodule A do
defmacro my_macro({:__aliases__, _, list}) do
module = Module.concat(list)
module.foo()
end
end
defmodule B do
def foo do
quote do
42
end
end
end
defmodule C do
import A
IO.inspect my_macro B
end
Output:
42

How to convert a string to a variable name?

I would like to have a construct like below to declare variable names based on a string queue. The below doesn't compile. So I would like to know if a similar approach is possible in Systemverilog.
Below is a simplified version of what I want to actually implement.
`define declare_int(NAME) int ``NAME`` = 1;
string str_q[$] = {"a", "b", "c"};
foreach (str_q[i]) begin
`declare_int(str_q[i])
end
NOTE: I am aware that `declare_int(a) will translate to int a = 1;. But as shown in the example above, I need a foreach loop to call this macro multiple times and so the input of the macro has to be some datatype, like a string in this case. The purpose is to auto-declare stuff as the queue changes with time.
In other words, how can I define the `declare_int macro so that `declare_int("a") translates to int a = 1;?
As Verilog is not interpreted but compiled in simulation, I doubt theres any way to dynamically declare variables at runtime. However, there are work arounds that achieve a similar effect.
I think the closest thing you could get is an associative array with the keys as your names (a, b, c) and your values for the values. For example, instead of your code, you'd have:
int vars[string];
string str_q[$] = {"a", "b", "c"};
foreach (str_q[i]) begin
vars[str_q[i]] = 1;
end
...
// To use the variable, just do:
vars["a"];
For more on associative arrays: http://www.asic-world.com/systemverilog/data_types13.html