"Too few positionals" in macro definition - macros

I'm trying to create some examples of using (experimental) macros this way:
use experimental :macros;
macro cards_vars() {
(<hearts clubs diamonds spades> X~ 1..10).map: { "my \$$^x = False;" }
};
cards_vars();
say $hearts1;
This creates and runs the macro, and then checks if one of the variable defined exists. But I get this error:
Too few positionals passed; expected 3 arguments but got 2
I don't even know where that error comes from. I think it's in cards_vars(), but I have no idea if that's the case or not. Declaring the macro without the parentheses yields the same error.

Update: To be clear, macros are not just experimental but very rudimentary and buggy. For a little more about this, see Moritz's answer to another macros question.
Let's start with a golf of your code:
use experimental :macros;
macro foo { 42 }
foo
This yields the same compile-time error:
Too few positionals passed; expected 3 arguments but got 2
You get this error if you return anything other than an AST object from a macro.1 Which is reasonable given that this is the entire point of the macro construct. The error message is less than awesome -- but then again macros are an experimental feature.
So you need to return an AST. Here's one way:
use experimental :macros;
macro foo { AST.new }
foo
This yields the run-time error:
Useless use of constant value foo in sink context (line 3)
This shows that the macro has done its job. The compiler has completed compilation and proceeded to run-time.
Explicitly returning AST objects is the wrong way to proceed. The first reason is given on the AST doc page:
There is no API defined for ASTs yet. Hopefully that will emerge as part of the work on macros.
The second reason is that there's a higher level construct for constructing AST objects, namely the quasi { ... } construct3 mentioned on the doc page and by Scimon:
use experimental :macros;
macro foo { quasi { 42 } }
say foo; # 42
The quasi block tells the compiler to compile the enclosed statements into AST form.
Note that the result isn't an AST corresponding to a block. In the above it's an AST object corresponding to 42.
So, finally, back to what you were trying to do.
Simplifying:
use experimental :macros;
macro foo { quasi { my $bar } }
foo;
say $bar
yields:
Error while compiling ... Variable '$bar' is not declared
Whereas:
use experimental :macros;
macro foo { quasi { class bar {} } }
foo;
say bar; # (bar)
So, bottom line, lexical declarations currently fail to stick.
The experiment hasn't gotten that far.
Carl Mäsak, the creator of the experimental macros in Rakudo, is continuing the experiment at 007. If you leave an issue at its GH repo Carl will answer.
1 When the compiler encounters a call to a routine that it knows is a macro2 it replaces the macro call with its result. In more detail the compiler:
Compiles all the macro's arguments into AST form;
Calls the macro (at compile-time) passing it the arguments in their AST form;
Runs the macro routine which is expected to return an AST object (er, "ASTish");
Splices the resulting AST(ish) object into the overall AST being constructed.
If you don't return an AST(ish) object the last step goes wrong with the error message in your title.
2 Currently, if you put the macro definition after the call then the compiler doesn't realize it's a macro and treats it as an ordinary routine:
use experimental :macros;
say foo; # 42
macro foo { 42 }
say bar; # AST.new
macro bar { quasi { 42 } }
say bar; # 42
It would presumably be better if this caused a compile-time error.
3 Carl, the macros champion, likes the word quasi. It draws from lisp heritage which draws from mathematics heritage. Imo it is a "too cute" choice that's a less than awesome fit for P6. My rationale is that Perl philosophy suggests choice of keywords with familiar meaning to newbies. The word "quasi" has a well known normal English connotation of meaning "sort of, but not really". That's not at all helpful. (I have suggested toAST to Carl but he prefers quasi.)

So a bit of poking about and I found this old halloween article : https://perl6advent.wordpress.com/2012/12/23/day-23-macros/
Which gives some pointer. You need to use quasi to highlight the code you're going to output. But getting it to create a variable? Not sure. I'm still setting if it's possible.

Related

How to evaluate an expression in nim?

I'm trying to perform the equivalent of the eval method from Python in nim.
I was under the impression that parseStmt from the macros package should help me with this, but I'm facing a compilation issue that I don't understand.
import macros
echo parseStmt("1 + 2")
I would have expected this to print 3 when executed, but instead the compilation complains that
Error: request to generate code for .compileTime proc: $
I found this thread, and the examples there work, and following this, I was able to make the following program that works as I would expect:
import macros
import strformat
macro eval(value: string): untyped =
result = parseStmt fmt"{value}"
echo eval("1+2")
But I don't undertand why it needs to be written in this way at all. If I inline the statement, let value = "1 + 2"; echo parseStmt fmt"{value}", I get the same compile error as above.
Also, why is parseStmt value different from parseStmt fmt"{value}", in the context of the eval macro above?
What am I missing here?
Thank you in advance for any clarifications!
Unlike Python which is an interpreted language, Nim is compiled. This means that all code is parsed and turned into machine code on compile-time and the program that you end up with doesn't really know anything about Nim at all (at least as long as you don't import the Nim compiler as a module, which is possible). So parseStmt and all macro/template expansion stuff in Nim is done completely during compilation. The error, although maybe a bit hard to read, is trying to tell you that what was passed to $ (which is the convert-to-string operator in Nim, and called by echo on all its arguments) is a compile-time thing that can't be used on runtime. In this case it's because parseStmt doesn't return "3", it returns something like NimNode(kind: nnkIntLit, intVal: 3), and the NimNode type is only available during compile-time. Nim however allows you to run code on compile-time to return other code, this is what a macro does. The eval macro you wrote there takes value which is any statement that resolves to a string during runtime, passed as a NimNode. This is also why result = parseStmt value won't work in your case, because value is not yet a string, but could be something like reading a string from standard input which would result in a string during runtime. However the use of strformat here is a bit confusing and overkill. If you change your code to:
import macros
macro eval(value: static[string]): untyped =
result = parseStmt value
echo eval("1+2")
It will work just fine. This is because we have now told Nim that value needs to be a static i.e. known during compile-time. In this case the string literal "1+2" is obviously known at compile-time, but this could also be a call to a compile-time procedure, or even the output of staticRead which reads a file during compilation.
As you can see Nim is very powerful, but the barrier between compile-time and run-time can sometimes be a bit confusing. Note also that your eval procedure doesn't actually evaluate anything at compile-time, it simply returns the Nim code 1 + 2 so your code ends up being echo 1 + 2. If you want to actually run the code at compile-time you might want to look into compile-time procedures.
Hope this helps shed some light on your issue.
Note: while this answer outlines why this happens, keep in mind that what you're trying to do probably won't result in what you want (which I assumed to be runtime evaluation of expressions).
You're trying to pass a NimNode to parseStmt which expects a string. The fmt macro automatically stringifies anything in the {}, you can omit the fmt by doing $value to turn the node into a string.
As I already noted, this will not work as it does in Python: Nim does not have runtime evaluation. The expression in the string is going to be evaluated at compile time, so a simple example like this will not do what you want:
import std/rdstdin
let x = readLineFromStdin(">")
echo eval(x)
Why?
First of all, because you're stringifying the AST you pass to the eval, it's not the string behind the x variable that's going to get passed to the macro - it's going to be the symbol that denotes the x variable. If you stringify a symbol, you get the underlying identifier, which means that parseStmt will receive "x" as its parameter. This will effect in the string stored in x being printed out, which is wrong.
What you want instead is the following:
import std/rdstdin
import std/macros
macro eval(value: static string): untyped =
result = parseStmt(value)
echo eval("1 + 2")
This prevents runtime-known values from being passed to the macro. You can only pass consts and literals to it now, which is the correct behavior.

Why do some users quote classnames in Perl?

Looking at Type::Tiny, I see that the class name in the call to Type::Tiny->new is quoted in the official docs,
my $NUM = "Type::Tiny"->new(
name => "Number",
constraint => sub { looks_like_number($_) },
message => sub { "$_ ain't a number" },
);
Why is this? Is this mere style? Is there any performance ramifications for this practice?
Take a simpler example
package Foo { sub new { die 7 } };
package Bar { sub new { die 42 } };
sub Foo { "Bar" }
Foo->new();
In the above example, the constant Foo resolves to "Bar", so so this calls "Bar"->new not "Foo"->new. How do you stop the subroutine from resolving? You can quote it.
"Foo"->new();
As for the performance implication, things are not made worse by using a string rather than a bareword. I've confirmed the optree generated by O=Deparse is the same. So as a general rule, it seems it's better to quote the Classnames if you value correctness.
This is mentioned in Programming Perl, (sadly in the context of indirect method invocation)
... so we'll tell you that you can almost always get away with a bare class name, provided two things are true. First, there is no subroutine of the same name as the class. (If you follow the convention that subroutine names like new start lowercase and class names like ElvenRing start uppercase, this is never a problem). Second, the class been loaded with one of
use ElvenRing;
require ElvenRing;
Either of these declarations ensures that Perl knows ElvenRing is a module name, which forces any bare name like new before the class name ElvenRing to be interpreted as a method call, even if you happen to have declare a new subroutine of your own in the current package.
And, that makes sense: the confusion here can only happen if your subroutines (typically lowercase) have the same name as a class (typically uppercase). This can only happen if you violate that naming convention above.
tldr; it is probably a good idea to quote your classnames, if you know them and you value correctness over clutter.
Side note: alternatively you can stop resolution of a bareword to a function by affixing a :: to the end of it, for example in the above Foo::->new.
Thanks to Ginnz on reddit for pointing this out to me, and to Toby Inkster for the comment (though it didn't make sense to me on first read).
Explicitly quoting the class name rather than using a bareword (which is treated as a string) is one of three ways to avoid syntactic ambiguity. The Invoking Class Methods section of the perlobj documentation explains.
Because Perl allows you to use barewords for package names and
subroutine names, it sometimes interprets a bareword’s meaning
incorrectly. For example, the construct Class->new() can be
interpreted as either 'Class'->new() or Class()->new() .In English,
that second interpretation reads as “call a subroutine named Class(),
then call new() as a method on the return value of Class().” If there
is a subroutine named Class() in the current namespace, Perl will
always interpret Class->new() as the second alternative: a call to
new() on the object returned by a call to Class().
See this odd case in action with the demo below.
#! /usr/bin/env perl
use strict;
use warnings;
sub Type::Tiny { print "Returning Bogus\n" ; return "Bogus" }
sub Type::Tiny::new { print "Type::Tiny::new\n" }
sub Bogus::new { print "Bogus::new\n" }
my $class = "Type::Tiny";
Type::Tiny->new;
Type::Tiny::->new;
"Type::Tiny"->new;
$class->new;
Its output is
Returning Bogus
Bogus::new
Type::Tiny::new
Type::Tiny::new
Type::Tiny::new
The rest of the aforementioned documentation section shows how to protect against surprising behavior or inadvertent errors.
You can force Perl to use the first interpretation (i.e., as a
method call on the class named "Class") in two ways. First, you can
append a :: to the class name:
Class::->new()
Perl will always interpret this as a method call.
Alternatively, you can quote the class name:
'Class'->new()
Of course, if the class name is in a scalar Perl will do the right
thing as well:
my $class = 'Class';
$class->new();
Applying to your question, all of the calls below are equivalent.
Type::Tiny::->new( … );
"Type::Tiny"->new( … );
my $class = "Type::Tiny";
$class->new( … );
Appending :: to the end has the advantage of producing a helpful warning. Say you accidentally typed
Type::Tinny::->new;
That produces
Bareword "Type::Tinny::" refers to nonexistent package at ./try line 15.
Can't locate object method "new" via package "Type::Tinny" (perhaps you forgot to load "Type::Tinny"?) at ./try line 15.

Matching against the enclosing function's return type

Is it possible for a Rust macro to match against the enclosing function's return type?
An example is something like a logging and assert macro which also returns Err in functions returning Result and panics in functions not returning Result. To implement this the macro should somehow know about the enclosing function's return type.
I suppose this is not possible with declarative macros (macro_rules!) because they have a limited set of matching types (as described in The Rust Reference, chapter Macros By Example): items, blocks, statements, patterns, expressions, types, identifiers, and so on, but not the enclosing function's return type.
But perhaps with procedural macros?
Summary: No, it's not easily possible, even with procedural macros. And I actually think you shouldn't write such a thing, even if it's possible. Just let your macro evaluate to a Result and let the user deal with it.
Function-like macro
Rust macros, procedural and declarative, only have access to their input stream: just a list of tokens. For function-like macros (the ones you invoke via foo!(...)), the input is just what you pass to them. So you could manually pass the return type:
macro_rules! foo {
(Result $($stuff:tt)*) => { return Err(()); };
($($stuff:tt)*) => { panic!(); };
}
fn returns_result() -> Result<String, ()> {
foo!(Result<(), ()>); // will return `Err`
Ok("hi".into())
}
fn returns_string() -> String {
foo!(String); // will panic
"hi".into()
}
But I guess that is not what you want: the user would have to manually specify the return type for each macro invocation.
The same goes for procedural macros that are invoked this way.
Procedural macro attribute
Can we define a procedural macro where the return type of the function is in the input token stream? Yes, preferably via a proc-macro attribute. If you define such an attribute bar, you could write this:
#[bar]
fn returns_result() -> Result<String, ()> { ... }
And your procedural macro would receive the whole function definition as input, including the return type. But what are you going to do with that information?
You can change the whole function as you like, so one idea would be to search for all foo!() macro invocations in the function and replace them with return Err or panic!() depending on the return type. That is: do the macro invocation step for your own macros via a procedural macro.
But I think this is a bad idea for several reasons. Most importantly, I don't think it's well defined when the compiler calls the procedural macro. So the compiler could attempt to invoke your foo!() macros before calling the procedural macro.
So it could work via procedural macro, but not in a way that is typical. So it's rather hacky.
What I think is the best solution
Lastly, how would I do it? Let your macro evaluate to a Result. Then the user can easily decide themselves what to do with it. If they return a Result, they just need to add ?. If they don't, they have the freedom to choose between .unwrap(), expect() and other ways to panic.
I understand why you are trying to do what you want to do (it's easier for the user and comfortable), but I think it's not a good idea. It probably comes down to "spooky action at a distance": what a macro in your function does suddenly depends on the return type of that function. That means when you change that, the whole semantics of the function change. This sounds like something you could shoot yourself in the foot with very easily. That's also probably the reason why it's not easy in Rust.

How can I reuse definition (AST) subtrees in a macro?

I am working in a Scala embedded DSL and macros are becoming a main tool for achieving my purposes. I am getting an error while trying to reuse a subtree from the incoming macro expression into the resulting one. The situation is quite complex, but (I hope) I have simplified it for its understanding.
Suppose we have this code:
val y = transform {
val x = 3
x
}
println(y) // prints 3
where 'transform' is the involved macro. Although it could seem it does absolutely nothing, it is really transforming the shown block into this expression:
3 match { case x => x }
It is done with this macro implementation:
def transform(c: Context)(block: c.Expr[Int]): c.Expr[Int] = {
import c.universe._
import definitions._
block.tree match {
/* {
* val xNam = xVal
* xExp
* }
*/
case Block(List(ValDef(_, xNam, _, xVal)), xExp) =>
println("# " + showRaw(xExp)) // prints Ident(newTermName("x"))
c.Expr(
Match(
xVal,
List(CaseDef(
Bind(xNam, Ident(newTermName("_"))),
EmptyTree,
/* xExp */ Ident(newTermName("x")) ))))
case _ =>
c.error(c.enclosingPosition, "Can't transform block to function")
block // keep original expression
}
}
Notice that xNam corresponds with the variable name, xVal corresponds with its associated value and finally xExp corresponds with the expression containing the variable. Well, if I print the xExp raw tree I get Ident(newTermName("x")), and that is exactly what is set in the case RHS. Since the expression could be modified (for instance x+2 instead of x), this is not a valid solution for me. What I want to do is to reuse the xExp tree (see the xExp comment) while altering the 'x' meaning (it is a definition in the input expression but will be a case LHS variable in the output one), but it launches a long error summarized in:
symbol value x does not exist in org.habla.main.Main$delayedInit$body.apply); see the error output for details.
My current solution consists on the parsing of the xExp to sustitute all the Idents with new ones, but it is totally dependent on the compiler internals, and so, a temporal workaround. It is obvious that the xExp comes along with more information that the offered by showRaw. How can I clean that xExp for allowing 'x' to role the case variable? Can anyone explain the whole picture of this error?
PS: I have been trying unsuccessfully to use the substitute* method family from the TreeApi but I am missing the basics to understand its implications.
Disassembling input expressions and reassembling them in a different fashion is an important scenario in macrology (this is what we do internally in the reify macro). But unfortunately, it's not particularly easy at the moment.
The problem is that input arguments of the macro reach macro implementation being already typechecked. This is both a blessing and a curse.
Of particular interest for us is the fact that variable bindings in the trees corresponding to the arguments are already established. This means that all Ident and Select nodes have their sym fields filled in, pointing to the definitions these nodes refer to.
Here is an example of how symbols work. I'll copy/paste a printout from one of my talks (I don't give a link here, because most of the info in my talks is deprecated by now, but this particular printout has everlasting usefulness):
>cat Foo.scala
def foo[T: TypeTag](x: Any) = x.asInstanceOf[T]
foo[Long](42)
>scalac -Xprint:typer -uniqid Foo.scala
[[syntax trees at end of typer]]// Scala source: Foo.scala
def foo#8339
[T#8340 >: Nothing#4658 <: Any#4657]
(x#9529: Any#4657)
(implicit evidence$1#9530: TypeTag#7861[T#8341])
: T#8340 =
x#9529.asInstanceOf#6023[T#8341];
Test#14.this.foo#8339[Long#1641](42)(scala#29.reflect#2514.`package`#3414.mirror#3463.TypeTag#10351.Long#10361)
To recap, we write a small snippet and then compile it with scalac, asking the compiler to dump the trees after the typer phase, printing unique ids of the symbols assigned to trees (if any).
In the resulting printout we can see that identifiers have been linked to corresponding definitions. For example, on the one hand, the ValDef("x", ...), which represents the parameter of the method foo, defines a method symbol with id=9529. On the other hand, the Ident("x") in the body of the method got its sym field set to the same symbol, which establishes the binding.
Okay, we've seen how bindings work in scalac, and now is the perfect time to introduce a fundamental fact.
If a symbol has been assigned to an AST node,
then subsequent typechecks will never reassign it.
This is why reify is hygienic. You can take a result of reify and insert it into an arbitrary tree (that possibly defines variables with conflicting names) - the original bindings will remain intact. This works because reify preserves the original symbols, so subsequent typechecks won't rebind reified AST nodes.
Now we're all set to explain the error you're facing:
symbol value x does not exist in org.habla.main.Main$delayedInit$body.apply); see the error output for details.
The argument of the transform macro contains both a definition and a reference to a variable x. As we've just learned, this means that the corresponding ValDef and Ident will have their sym fields synchronized. So far, so good.
However unfortunately the macro corrupts the established binding. It recreates the ValDef, but doesn't clean up the sym field of the corresponding Ident. Subsequent typecheck assigns a fresh symbol to the newly created ValDef, but doesn't touch the original Ident that is copied to the result verbatim.
After the typecheck, the original Ident points to a symbol that no longer exists (this is exactly what the error message was saying :)), which leads to a crash during bytecode generation.
So how do we fix the error? Unfortunately there is no easy answer.
One option would be to utilize c.resetLocalAttrs, which recursively erases all symbols in a given AST node. Subsequent typecheck will then reestablish the bindings granted that the code you generated doesn't mess with them (if, for example, you wrap xExp in a block that itself defines a value named x, then you're in trouble).
Another option is to fiddle with symbols. For example, you could write your own resetLocalAttrs that only erases corrupted bindings and doesn't touch the valid ones. You could also try and assign symbols by yourself, but that's a short road to madness, though sometimes one is forced to walk it.
Not cool at all, I agree. We're aware of that and intend to try and fix this fundamental issue sometimes. However right now our hands are full with bugfixing before the final 2.10.0 release, so we won't be able to address the problem in the nearest future. upd. See https://groups.google.com/forum/#!topic/scala-internals/rIyJ4yHdPDU for some additional information.
Bottom line. Bad things happen, because bindings get messed up. Try resetLocalAttrs first, and if it doesn't work, prepare yourself for a chore.

Is there an andand for Perl?

I'd rather do this:
say $shop->ShopperDueDate->andand->day_name();
vs. this:
say $shop->ShopperDueDate->day_name() if $shop->ShopperDueDate;
Any ideas?
(This idea is inspired by the Ruby andand extension.)
(Actually it is inspired by the Groovy language, but most people don't know that ;-)
update: I think that both maybe() and eval {} are good solutions. This isn't ruby so I can't expect to read all of the methods/functions from left to right anyway, so maybe could certainly be a help. And then of course eval really is the perl way to do it.
You can use Perl's eval statement to catch exceptions, including those from trying to call methods on an undefined argument:
eval {
say $shop->ShopperDueDate->day_name();
};
Since eval returns the last statement evaluated, or undef on failure, you can record the day name in a variable like so:
my $day_name = eval { $shop->ShopperDueDate->day_name(); };
If you actually wish to inspect the exception, you can look in the special variable $#. This will usually be a simple string for Perl's built-in exceptions, but may be a full exception object if the exception originates from autodie or other code that uses object exceptions.
eval {
say $shop->ShopperDueDate->day_name();
};
if ($#) {
say "The error was: $#";
}
It's also possible to string together a sequence of commands using an eval block. The following will only check to see if it's a weekend provided that we haven't had any exceptions thrown when looking up $day_name.
eval {
my $day_name = $shop->ShopperDueDate->day_name();
if ($day_name ~~ [ 'Saturday', 'Sunday' ] ) {
say "I like weekends";
}
};
You can think of eval as being the same as try from other languages; indeed, if you're using the Error module from the CPAN then you can even spell it try. It's also worth noting that the block form of eval (which I've been demonstrating above) doesn't come with performance penalties, and is compiled along with the rest of your code. The string form of eval (which I have not shown) is a different beast entirely, and should be used sparingly, if at all.
eval is technically considered to be a statement in Perl, and hence is one of the few places where you'll see a semi-colon at the end of a block. It's easy to forget these if you don't use eval regularly.
Paul
I assume your second example should better be:
say $shop->ShopperDueDate ? $shop->ShopperDueDate->day_name() : undef;
instead of what it actually says.
Anyway, you can't do it with exactly that syntax without a source filter because undef isn't an object, but an unary operator, so it can't have any methods.
Instead, consider this:
package noop;
our $maybe = bless [], 'noop';
sub AUTOLOAD { undef };
This $noop::maybe is an object; all of its methods return undef though. Elsewhere, you'll have a regular function like this:
sub maybe { $_[0] || $noop::maybe; }
Then you can write this:
say maybe($shop->ShopperDueDate)->day_name()
This works because "maybe" returns returns its argument if true, otherwise it returns our $noop::maybe object which has methods which always return undef.
EDIT: Correction! the ->andand-> syntax can be had without a source filter, by using an XS that mucks about with Perl's internals. Leon Timmermans made an implementation that takes this route. It replaces the undef() function globally, so it's likely to be a lot slower than my method.
I think I've just written it. I've just uploaded it to CPAN, you can find it here.
Something like (untested):
use Object::Generic::False;
sub UNIVERSAL::andand { $_[0] || Object::Generic::False->false }
UNIVERSAL is automatically a subclass of all other classes, so this provides andand for all objects.
(Obviously creating methods in UNIVERSAL has the potential for conflicts or surprising action at a distance,
so it shouldn't be used carelessly.) If the object upon which the andand method is called is true, return it;
otherwise return a generic false object which returns itself for any method call used upon it.