Elixir macros, quoting bitstring patternmatch types - macros

I am working on a project in which I am likely to be writing a lot of code of the form:
defmodule Kind
defstruct [name1, name2, name3]
#type t :: %Kind{name1: integer(), name2: integer(), name3: binary()}
def unpack(input) do
with <<name1::integer-8>> <- Enum.take(input, 1),
<<name2::integer-little-32>> <- Enum.take(input, 4),
<<name3::binary-10>> <- Enum.take(input, 10),
do: %Kind{name1: name1, name2: name2, name3: name3>>
end
end
(for arbitrary sets of input names and types, input being a binary stream producing one byte at a time)
It would be very useful to be able to handle this in a macro, so that I could simply write (for example) use Unpack quote([{name1, integer-8}, {name2, integer-little-32}, {name3, binary-10}]) and automatically generate the necessary struct, typedef, and unpacking function, for arbitrary named fields of fixed sizes. Could even expand it, add a third field in the tuples to pass a function to handle variably-sized types. Unfortunately, when I try to do a simpler version of that (only taking one sized field, and only matching 1 to it):
defmodule Unpack do
defmacro testmacro({name, kind}) do
quote do
<<unquote(name)::unqote(kind)>> = 1
end
end
end
The system tells me it's got invalid arguments for quote/1. I assume this is because the "types" used in bitstring pattern-matching are a special form, as are bitstring literals in general, and those particular items are not used anywhere else.
So, how do I get around that? I've got more than a dozen kinds of packed struct to unpack, each with between five and twenty different fields. If I don't do this I'll probably resort to Vim macros to at least save my hands... but that won't really help with having large amounts of extremely repetitive code to maintain.

Two things: you have a typo in unquote and the RHS must be a binary so that the pattern matches. With those changes, your code works for me:
defmodule Unpack do
defmacro unpack({name, kind}) do
quote do
<<unquote(name)::unquote(kind)>> = "a"
end
end
end
defmodule Main do
import Unpack
def main do
unpack({foo, integer-8})
IO.inspect foo
end
end
Main.main
Output:
97

Related

kdb: differences between value and eval

From KX: https://code.kx.com/q/ref/value/ says, when x is a list, value[x] will be result of evaluating list as a parse tree.
Q1. In code below, I understand (A) is a parse tree, given below definition. However, why does (B) also work? Is ("+";3;4) a valid parse tree?
q)value(+;3;4) / A
7
q)value("+";3;4) / B
7
q)eval(+;3;4) / C
7
q)eval("+";3;4) / D
'length
[0] eval("+";3;4)
Any other parse tree takes a form of a list, of which the first item
is a function and the remaining items are its arguments. Any of these
items can be parse trees. https://code.kx.com/q/basics/parsetrees/
Q2. In below code, value failed to return the result of what I think is a valid parse tree, but eval works fine, recursively evaluating the tree. Does this mean the topmost description is wrong?
q)value(+;3;(+;4;5))
'type
[0] value(+;3;(+;4;5))
^
q)eval(+;3;(+;4;5))
12
Q3. In general then, how do we choose whether to use value or eval?
put simply the difference between eval and value is that eval is specifically designed to evaluate parse trees, whereas value works on parse trees among other operations it does. For example value can be used to see the non-keyed values of dictionaries, or value strings, such as:
q)value"3+4"
7
Putting this string instead into the eval, we simply get the string back:
q)eval"3+4"
"3+4"
1 Following this, the first part of your question isn't too bad to answer. The format ("+";3;4) is not technically the parsed form of 3+4, we can see this through:
q)parse"3+4"
+
3
4
The good thing about value in this case is that it is valuing the string "+" into a the operator + and then valuing executing the parse tree. eval cannot understand the string "+" as this it outside the scope of the function. Which is why A, B and C work but not D.
2 In part two, your parse tree is indeed correct and once again we can see this with the parse function:
q)parse"3+(4+5)"
+
3
(+;4;5)
eval can always be used if your parse tree represents a valid statement to get the result you want. value will not work on all parse tree's only "simple" ones. So the nested list statement you have here cannot be evaluated by value.
3 In general eval is probably the best function of choice for evaluating your parse trees if you know them to be the correct parse tree format, as it can properly evaluate your statements, even if they are nested.

Elixir macros - pattern match on an argument passed as a variable

I'm trying to get a better understanding of Elixir macros, but I'm having trouble figuring out how to set up a scenario where I can pattern match the argument that I pass to a macro when the value I'm trying to pass is a variable. Here's a simple example to illustrate:
macro_test.ex
defmodule MacroTest do
use MacroTest.UseMe
def run() do
atom = :hello_world
macro_a(atom)
end
end
use_me.ex
defmodule MacroTest.UseMe do
defmacro __using__(_) do
quote do
defmacro macro_a(:hello_world) do
quote do
"Success!"
end
end
defmacro macro_a(:some_other_atom) do
quote do
"Did something else..."
end
end
end
end
end
When I try to compile this code, I get the error
== Compilation error in file lib/macro_test.ex ==
** (FunctionClauseError) no function clause matching in MacroTest.macro_a/1
If I change the initial run() function so that the atom is passed directly to the macro, such as macro_a(:hello_world), then it compiles/runs just fine.
How can I alter this scenario so that the macro can pattern match on a value that has been provided as a variable, rather than the literal value itself?
It depends on what you want to do. Macros run at compile time, and operate on portions of source code (as opposed to the run time value returned by said piece of source code). So in this case, the macro knows that it has been called with a variable called atom, but it has no way of knowing that this variable has been assigned to earlier in the calling function.
You could check the value of the given variable at run time:
defmacro macro_a(a) do
quote do
case unquote(a) do
:hello_world ->
"Success!"
:some_other_atom ->
"Did something else..."
end
end
end
That is, every invocation of macro_a gets replaced by the case expression above, using whatever variable or other expression was passed as argument.

Invoke a private macro within a quote block

I'm trying to invoke a private macro, within a quote block, using a variable defined within the code block itself.
This is the pseudo-code showing what I would like to do (doesn't work)
defmodule Foo do
defmacrop debug(msg) do
quote bind_quoted: [msg: msg], do: IO.puts(msg)
end
defmacro __using__(_) do
quote do
def hello do
my = "testme"
unquote(debug(quote do: my))
end
end
end
end
defmodule Bar do
use Foo
end
Bar.hello()
And this would get converted (in my mind), at compile time to:
defmodule Bar do
def hello do
my = "testme"
IO.puts(my)
end
end
Is there any way to achieve this? I'm struggling to find any documentation related to it.
Update
I discovered that:
defmodule Foo do
defmacrop debug() do
quote do: IO.puts("hello")
end
defmacro __using__(_) do
quote do
def hello do
my = "testme"
unquote(debug())
end
end
end
end
Gets properly converted to what I need, but I'm struggling find a way to pass the variable as is, so that it becomes IO.puts(my)
The issue here is with nested quoting: the private macro should return the double-quoted expression (since to invoke it from the outer scope one needs to explicitly unquote, and macro is still expected to return a quoted expression.)
Sidenote: your update section is wrong; you might notice, that "hello" is printed during a compilation stage, namely when use Foo is being compiled. That is because the double-quoting is needed, the code in your update section executes IO.puts when unquote in __using__ macro is met.
On the other hand, my should be quoted only once. That might be achieved with an explicit quoting of AST, passing the msg there as is:
defmodule Foo do
defmacrop debug(msg) do
quote bind_quoted: [msg: msg] do
{
{:., [], [{:__aliases__, [alias: false], [:IO]}, :puts]},
[],
[msg]} # ⇐ HERE `msg` is the untouched argument
end
end
defmacro __using__(_) do
quote do
def hello do
my = "testme"
unquote(debug(quote do: my))
end
end
end
end
defmodule Bar do
use Foo
end
Bar.hello()
#⇒ "testme"
I was unable to achieve the same functionality with options in the call to Kernel.SpecialForms.quote/2; the only available related option is unquote to tune the unquoting inside nested quotes, while we need the exact opposite.
Sidenote: below does not work and I expect this to be a bug in Kernel.SpecialForms.quote/2 implementation.
quote bind_quoted: [msg: msg] do
quote bind_quoted: [msg: msg], do: IO.puts(msg)
end
FWIW: I filed an issue.
I believe it might be a good feature request to Elixir core, to allow an option that disables additional quoting.
Sidenote 2: the following works (most concise approach):
defmacrop debug(msg) do
quote bind_quoted: [msg: msg] do
quote do: IO.puts(unquote msg)
end
end
So you might avoid tackling with an explicit AST and just use the above. I am leaving the answer as is, since dealing with AST directly is also a very good option, that should be used as a sledgehammer / last resort, which does always work.
If IO.puts is not your desired target, you might call quote do: YOUR_EXPR on what you want to have in debug macro:
quote do: to_string(arg)
#⇒ {:to_string, [context: Elixir, import: Kernel], [{:arg, [], Elixir}]}
and manually unquote the arg in the result:
# ✗ ⇓⇓⇓ {:arg, [], Elixir}
# ✓ ⇓⇓⇓ arg
{:to_string, [context: Elixir, import: Kernel], [arg]}
This is basically how I got the AST of your original request (IO.puts.)

How do I write a perl6 macro to enquote text?

I'm looking to create a macro in P6 which converts its argument to a string.
Here's my macro:
macro tfilter($expr) {
quasi {
my $str = Q ({{{$expr}}});
filter-sub $str;
};
}
And here is how I call it:
my #some = tfilter(age < 50);
However, when I run the program, I obtain the error:
Unable to parse expression in quote words; couldn't find final '>'
How do I fix this?
Your use case, converting some code to a string via a macro, is very reasonable. There isn't an established API for this yet (even in my head), although I have come across and thought about the same use case. It would be nice in cases such as:
assert a ** 2 + b ** 2 == c ** 2;
This assert statement macro could evaluate its expression, and if it fails, it could print it out. Printing it out requires stringifying it. (In fact, in this case, having file-and-line information would be a nice touch also.)
(Edit: 007 is a language laboratory to flesh out macros in Perl 6.)
Right now in 007 if you stringify a Q object (an AST), you get a condensed object representation of the AST itself, not the code it represents:
$ bin/007 -e='say(~quasi { 2 + 2 })'
Q::Infix::Addition {
identifier: Q::Identifier "infix:+",
lhs: Q::Literal::Int 2,
rhs: Q::Literal::Int 2
}
This is potentially more meaningful and immediate than outputting source code. Consider also the fact that it's possible to build ASTs that were never source code in the first place. (And people are expected to do this. And to mix such "synthetic Qtrees" with natural ones from programs.)
So maybe what we're looking at is a property on Q nodes called .source or something. Then we'd be able to do this:
$ bin/007 -e='say((quasi { 2 + 2 }).source)'
2 + 2
(Note: doesn't work yet.)
It's an interesting question what .source ought to output for synthetic Qtrees. Should it throw an exception? Or just output <black box source>? Or do a best-effort attempt to turn itself into stringified source?
Coming back to your original code, this line fascinates me:
my $str = Q ({{{$expr}}});
It's actually a really cogent attempt to express what you want to do (turn an AST into its string representation). But I doubt it'll ever work as-is. In the end, it's still kind of based on a source-code-as-strings kind of thinking à la C. The fundamental issue with it is that the place where you put your {{{$expr}}} (inside of a string quote environment) is not a place where an expression AST is able to go. From an AST node type perspective, it doesn't typecheck because expressions are not a subtype of quote environments.
Hope that helps!
(PS: Taking a step back, I think you're doing yourself a disservice by making filter-sub accept a string argument. What will you do with the string inside of this function? Parse it for information? In that case you'd be better off analyzing the AST, not the string.)
(PPS: Moritz++ on #perl6 points out that there's an unrelated syntax error in age < 50 that needs to be addressed. Perl 6 is picky about things being defined before they are used; macros do not change this equation much. Therefore, the Perl 6 parser is going to assume that age is a function you haven't declared yet. Then it's going to consider the < an opening quote character. Eventually it'll be disappointed that there's no >. Again, macros don't rescue you from needing to declare your variables up-front. (Though see #159 for further discussion.))

Is it possible to influence how '_' lambda arguments are ordered?

I have a statement which currently looks like this:
arrays.foldLeft(0)((offset, array) => array.copyTo(largerArray, offset))
It would be great to express it as follows, but this is not possible since foldLeft is designed to take the seed argument first:
arrays.foldLeft(0)(_.copyTo(largerArray, _))
This is purely superficial - I'm just curious!
p.s. copyTo returns the next offset in this example.
The SLS seems to say "no".
Section 6.23, Anonymous Functions/Placeholder Syntax for Anonymous Functions:
An expression (of syntactic category Expr) may contain embedded
underscore symbols _ at places where identifiers are legal. Such an
expression represents an anonymous function where subsequent
occurrences of underscores denote successive parameters.
and
If an expression e binds underscore sections u1 , . . . , un, in this order, it is equivalent to the anonymous function (u'1 , ... u'n ) => e' where each u'i results from ui by replacing the
underscore with a fresh identifier and e' results from e by
replacing each underscore section ui by u'i.
Emphasis is mine - it explicitly states in both relevant section that a preserved ordering is assumed.
Personally, I think it makes sense to enforce that, if "only" for readability reasons.