Assuming I have a method like the following:
def getInfo(func: () => T) = {
//Code goes here.
}
How could I use the runtime reflection of Scala 2.11.1 to inspect the passed anonymous function func?
I'm especially interested in getting an AST (abstract syntax tree) of func and, if possible, the location (line number, file) where the method was first defined.
All I have accomplished so far is to get information about the type of parameter func, not the function itself.
I am aware of the fact that there have been similar questions on SO, but they mainly target other Scala versions.
As Ben mentioned in the comments, this can be done using Scala macros at compile time.
A possible option is to expand the original call to a macro, which queries the necessary information, and then call a internal getInfoMethod, which does something with the information.
Example for getInfo function:
import scala.language.experimental.macros
//getInfo method which gets expanded to macro
def getInfo(func: => Any):Unit = macro FindFreeVars.findMacro
def getInfoInternal(info: Any) {
//Do something with the collected information
}
Example for macro:
//Macro delclaration
def getInfoMacro(c: Context)(func: c.Tree): c.Expr[Unit] = {
import c.universe._
//Extract information, like enclosingPosition or symols from the function tree.
val functionInfo = getFunctionInfo(func)
//Call internal method
c.Expr[List[(String, Any)]](q"getInfoInternal($func, $closedVars)")
}
More on Symbols, Trees and Types for analysis.
More complete example of a macro which finds all variables bound from another scope in a function by applying the same technique.
Related
New to Scala and looking for pointers to an idiomatic solution, if there is one.
I'd like to have arbitrary user-supplied Scala functions (which are allowed to reference functions/classes I have defined in my code) applied to some data.
For example: I have foo(s: String): String and bar(s: String): String functions defined in my myprog.scala. The user runs my program like this:
$ scala myprog data.txt --func='(s: Str) => foo(bar(s)).reverse'
This would run line by line through the data file and emit the result of applying the user-specified function to that line.
For extra points, can I ensure that there are no side-effects in the user-defined function? If not, can I restrict the function to use only a restricted subset of functions (which I can assure to be safe)?
#kenjiyoshida has a nice gist that shows how to eval Scala code. Note that when using Eval from that gist, not specifying a return value will result in a runtime failure when Scala defaults to inferring Nothing.
scala> Eval("println(\"Hello\")")
Hello
java.lang.ClassCastException: scala.runtime.BoxedUnit cannot be cast to scala.runtime.Nothing$
... 42 elided
vs
scala> Eval[Unit]("println(\"Hello\")")
Hello
It nicely handles whatever's in scope as well.
object Thing {
val thing: Int = 5
}
object Eval {
def apply[A](string: String): A = {
val toolbox = currentMirror.mkToolBox()
val tree = toolbox.parse(string)
toolbox.eval(tree).asInstanceOf[A]
}
def fromFile[A](file: File): A =
apply(scala.io.Source.fromFile(file).mkString(""))
def fromFileName[A](file: String): A =
fromFile(new File(file))
}
object Thing2 {
val thing2 = Eval[Int]("Thing.thing") // 5
}
Twitter's util package used to have util-eval, but that seems to have been deprecated now (and also triggers a compiler bug when compiled).
As for the second part of your question, the answer seems to be no. Even if you disable default Predef and imports yourself, a user can always get to those functions with the fully qualified package name. You could perhaps use Scala's scala.tools.reflect.ToolBox to first parse your string and then compare against a whitelist, before passing to eval, but at that point things could get pretty hairy since you'll be manually writing code to sanitize the Scala AST (or at the very least reject dangerous input). It definitely doesn't seem to be an "idiomatic solution."
This should be possible by using the standard Java JSR 223 Scripting Engine
see https://issues.scala-lang.org/browse/SI-874
(also mentions using scala.tools.nsc.Interpreter but not sure this is still available)
import javax.script.*;
ScriptEngine e = new ScriptEngineManager().getEngineByName("scala");
e.getContext().setAttribute("label", new Integer(4), ScriptContext.ENGINE_SCOPE);
try {
engine.eval("println(2+label)");
} catch (ScriptException ex) {
ex.printStackTrace();
}
I have a following function:
def getIntValue(x: Int)(implicit y: Int ) : Int = {x + y}
I see above declaration everywhere. I understand what above function is doing. It is a currying function which takes two arguments. If you omit the second argument, it will invoke implicit definition which returns int instead. So I think it is something very similar to defining a default value for the argument.
implicit val temp = 3
scala> getIntValue(3)
res8: Int = 6
I was wondering what are the benefits of above declaration?
Here's my "pragmatic" answer: you typically use currying as more of a "convention" than anything else meaningful. It comes in really handy when your last parameter happens to be a "call by name" parameter (for example: : => Boolean):
def transaction(conn: Connection)(codeToExecuteInTransaction : => Boolean) = {
conn.startTransaction // start transaction
val booleanResult = codeToExecuteInTransaction //invoke the code block they passed in
//deal with errors and rollback if necessary, or commit
//return connection to connection pool
}
What this is saying is "I have a function called transaction, its first parameter is a Connection and its second parameter will be a code-block".
This allows us to use this method like so (using the "I can use curly brace instead of parenthesis rule"):
transaction(myConn) {
//code to execute in a transaction
//the code block's last executable statement must be a Boolean as per the second
//parameter of the transaction method
}
If you didn't curry that transaction method, it would look pretty unnatural doing this:
transaction(myConn, {
//code block
})
How about implicit? Yes it can seem like a very ambiguous construct, but you get used to it after a while, and the nice thing about implicit functions is they have scoping rules. So this means for production, you might define an implicit function for getting that database connection from the PROD database, but in your integration test you'll define an implicit function that will superscede the PROD version, and it will be used to get a connection from a DEV database instead for use in your test.
As an example, how about we add an implicit parameter to the transaction method?
def transaction(implicit conn: Connection)(codeToExecuteInTransaction : => Boolean) = {
}
Now, assuming I have an implicit function somewhere in my code base that returns a Connection, like so:
def implicit getConnectionFromPool() : Connection = { ...}
I can execute the transaction method like so:
transaction {
//code to execute in transaction
}
and Scala will translate that to:
transaction(getConnectionFromPool) {
//code to execute in transaction
}
In summary, Implicits are a pretty nice way to not have to make the developer provide a value for a required parameter when that parameter is 99% of the time going to be the same everywhere you use the function. In that 1% of the time you need a different Connection, you can provide your own connection by passing in a value instead of letting Scala figure out which implicit function provides the value.
In your specific example there are no practical benefits. In fact using implicits for this task will only obfuscate your code.
The standard use case of implicits is the Type Class Pattern. I'd say that it is the only use case that is practically useful. In all other cases it's better to have things explicit.
Here is an example of a typeclass:
// A typeclass
trait Show[a] {
def show(a: a): String
}
// Some data type
case class Artist(name: String)
// An instance of the `Show` typeclass for that data type
implicit val artistShowInstance =
new Show[Artist] {
def show(a: Artist) = a.name
}
// A function that works for any type `a`, which has an instance of a class `Show`
def showAListOfShowables[a](list: List[a])(implicit showInstance: Show[a]): String =
list.view.map(showInstance.show).mkString(", ")
// The following code outputs `Beatles, Michael Jackson, Rolling Stones`
val list = List(Artist("Beatles"), Artist("Michael Jackson"), Artist("Rolling Stones"))
println(showAListOfShowables(list))
This pattern originates from a functional programming language named Haskell and turned out to be more practical than the standard OO practices for writing a modular and decoupled software. The main benefit of it is it allows you to extend the already existing types with new functionality without changing them.
There's plenty of details unmentioned, like syntactic sugar, def instances and etc. It is a huge subject and fortunately it has a great coverage throughout the web. Just google for "scala type class".
There are many benefits, outside of your example.
I'll give just one; at the same time, this is also a trick that you can use on certain occasions.
Imagine you create a trait that is a generic container for other values, like a list, a set, a tree or something like that.
trait MyContainer[A] {
def containedValue:A
}
Now, at some point, you find it useful to iterate over all elements of the contained value.
Of course, this only makes sense if the contained value is of an iterable type.
But because you want your class to be useful for all types, you don't want to restrict A to be of a Seq type, or Traversable, or anything like that.
Basically, you want a method that says: "I can only be called if A is of a Seq type."
And if someone calls it on, say, MyContainer[Int], that should result in a compile error.
That's possible.
What you need is some evidence that A is of a sequence type.
And you can do that with Scala and implicit arguments:
trait MyContainer[A] {
def containedValue:A
def aggregate[B](f:B=>B)(implicit ev:A=>Seq[B]):B =
ev(containedValue) reduce f
}
So, if you call this method on a MyContainer[Seq[Int]], the compiler will look for an implicit Seq[Int]=>Seq[B].
That's really simple to resolve for the compiler.
Because there is a global implicit function that's called identity, and it is always in scope.
Its type signature is something like: A=>A
It simply returns whatever argument is passed to it.
I don't know how this pattern is called. (Can anyone help out?)
But I think it's a neat trick that comes in handy sometimes.
You can see a good example of that in the Scala library if you look at the method signature of Seq.sum.
In the case of sum, another implicit parameter type is used; in that case, the implicit parameter is evidence that the contained type is numeric, and therefore, a sum can be built out of all contained values.
That's not the only use of implicits, and certainly not the most prominent, but I'd say it's an honorable mention. :-)
Methods are often declared with obvious parameter names, e.g.
def myMethod(s: String, image: BufferedImage, mesh: Mesh) { ... }
Parameter names correspond to parameter types.
1) "s" is often used for String
2) "i" for Int
3) lowercased class name for one word named classes (Mesh -> mesh)
4) lowercased last word from class name for long class names (BufferedImage -> image)
(Of course, it would not be convenient for ALL methods and arguments. Of course, somebody would prefer other rules…)
Scala macros are intended to generate some expressions in code. I would like to write some specific macros to convert to correct Scala expressions something like this:
// "arguments interpolation" style
// like string interpolation
def myMethod s[String, BufferedImage, Mesh]
{ /* code using vars "s", "image", "mesh" */ }
// or even better:
mydef myMethod[String, BufferedImage, Mesh]
{ /* code using vars "s", "image", "mesh" */ }
Is it possible?
Currently it is not possible and probably it will never be. Macros can not introduce their own syntax - they must be represented through valid Scala code (which can be executed at compile time) and, too, they must generate valid Scala code (better say a valid Scala AST).
Both of your shown examples are not valid Scala code, thus Macros can not handle them. Nevertheless, the current nightly build of Macro Paradise includes untyped macros. They allow to write Scala code which is typechecked after they are expanded, this means it is possible to write:
forM({i = 0; i < 10; i += 1}) {
println(i)
}
Notice, that the curly braces inside of the first parameter list are needed because, although the code is not typechecked when one writes it, it must represent a valid Scala AST.
The implementation of this macro looks like this:
def forM(header: _)(body: _) = macro __forM
def __forM(c: Context)(header: c.Tree)(body: c.Tree): c.Tree = {
import c.universe._
header match {
case Block(
List(
Assign(Ident(TermName(name)), Literal(Constant(start))),
Apply(Select(Ident(TermName(name2)), TermName(comparison)), List(Literal(Constant(end))))
),
Apply(Select(Ident(TermName(name3)), TermName(incrementation)), List(Literal(Constant(inc))))
) =>
// here one can generate the behavior of the loop
// but omit full implementation for clarity now ...
}
}
Instead of an already typechecked expression, the macro expects only a tree, that is typechecked after the expansion. The method call itself expects two parameter lists, whose parameter types can be delayed after the expansion phase if one uses an underscore.
Currently there is a little bit of documentation available but because it is extremely beta a lot of things will probably change in future.
With type macros it is possible to write something like this:
object O extends M {
// traverse the body of O to find what you want
}
type M(x: _) = macro __M
def __M(c: Context)(x: c.Tree): c.Tree = {
// omit full implementation for clarity ...
}
This is nice in order to delay the typechecking of the whole body because it allows to to cool things...
Macros that can change Scalas syntax are not planned at the moment and are probably not a good idea. I can't say if they will happen one day only future can tell us this.
Aside from the "why" (no really, why do you want to do that?), the answer is no, because as far as I know macros cannot (in their current state) generate methods or types, only expressions.
I've been working with Scala Macros and have the following code in the macro:
val fieldMemberType = fieldMember.typeSignatureIn(objectType) match {
case NullaryMethodType(tpe) => tpe
case _ => doesntCompile(s"$propertyName isn't a field, it must be another thing")
}
reify{
new TypeBuilder() {
type fieldType = fieldMemberType.type
}
}
As you can see, I've managed to get a c.universe.Type fieldMemberType. This represents the type of certain field in the object. Once I get that, I want to create a new TypeBuilder object in the reify. TypeBuilder is an abstract class with an abstract parameter. This abstract parameter is fieldType. I want this fieldType to be the type that I've found before.
Running the code shown here returns me a fieldMemberType not found. Is there any way that I can get the fieldMemberType to work inside the reify clause?
The problem is that the code you pass to reify is essentially going to be placed verbatim at the point where the macro is being expanded, and fieldMemberType isn't going to mean anything there.
In some cases you can use splice to sneak an expression that you have at macro-expansion time into the code you're reifying. For example, if we were trying to create an instance of this trait:
trait Foo { def i: Int }
And had this variable at macro-expansion time:
val myInt = 10
We could write the following:
reify { new Foo { def i = c.literal(myInt).splice } }
That's not going to work here, which means you're going to have to forget about nice little reify and write out the AST by hand. You'll find this happens a lot, unfortunately. My standard approach is to start a new REPL and type something like this:
import scala.reflect.runtime.universe._
trait TypeBuilder { type fieldType }
showRaw(reify(new TypeBuilder { type fieldType = String }))
This will spit out several lines of AST, which you can then cut and paste into your macro definition as a starting point. Then you fiddle with it, replacing things like this:
Ident(TypeBuilder)
With this:
Ident(newTypeName("TypeBuilder"))
And FINAL with Flag.FINAL, and so on. I wish the toString methods for the AST types corresponded more exactly to the code it takes to build them, but you'll pretty quickly get a sense of what you need to change. You'll end up with something like this:
c.Expr(
Block(
ClassDef(
Modifiers(Flag.FINAL),
anon,
Nil,
Template(
Ident(newTypeName("TypeBuilder")) :: Nil,
emptyValDef,
List(
constructor(c),
TypeDef(
Modifiers(),
newTypeName("fieldType"),
Nil,
TypeTree(fieldMemberType)
)
)
)
),
Apply(Select(New(Ident(anon)), nme.CONSTRUCTOR), Nil)
)
)
Where anon is a type name you've created in advance for your anonymous class, and constructor is a convenience method I use to make this kind of thing a little less hideous (you can find its definition at the end of this complete working example).
Now if we wrap this expression up in something like this, we can write the following:
scala> TypeMemberExample.builderWithType[String]
res0: TypeBuilder{type fieldType = String} = $1$$1#fb3f1f3
So it works. We've taken a c.universe.Type (which I get here from the WeakTypeTag of the type parameter on builderWithType, but it will work in exactly the same way with any old Type) and used it to define the type member of our TypeBuilder trait.
There is a simpler approach than tree writing for your use case. Indeed I use it all the time to keep trees at bay, as it can be really difficult to program with trees. I prefer to compute types and use reify to generate the trees. This makes much more robust and "hygienic" macros and less compile time errors. IMO using trees must be a last resort, only for a few cases, such as tree transforms or generic programming for a family of types such as tuples.
The tip here is to define a function taking as type parameters, the types you want to use in the reify body, with a context bound on a WeakTypeTag. Then you call this function by passing explicitly the WeakTypeTags you can build from universe Types thanks to the context WeakTypeTag method.
So in your case, that would give the following.
val fieldMemberType: Type = fieldMember.typeSignatureIn(objectType) match {
case NullaryMethodType(tpe) => tpe
case _ => doesntCompile(s"$propertyName isn't a field, it must be another thing")
}
def genRes[T: WeakTypeTag] = reify{
new TypeBuilder() {
type fieldType = T
}
}
genRes(c.WeakTypeTag(fieldMemberType))
I never understood it from the contrived unmarshalling and verbing nouns ( an AddTwo class has an apply that adds two!) examples.
I understand that it's syntactic sugar, so (I deduced from context) it must have been designed to make some code more intuitive.
What meaning does a class with an apply function give? What is it used for, and what purposes does it make code better (unmarshalling, verbing nouns etc)?
how does it help when used in a companion object?
Mathematicians have their own little funny ways, so instead of saying "then we call function f passing it x as a parameter" as we programmers would say, they talk about "applying function f to its argument x".
In mathematics and computer science, Apply is a function that applies
functions to arguments.
Wikipedia
apply serves the purpose of closing the gap between Object-Oriented and Functional paradigms in Scala. Every function in Scala can be represented as an object. Every function also has an OO type: for instance, a function that takes an Int parameter and returns an Int will have OO type of Function1[Int,Int].
// define a function in scala
(x:Int) => x + 1
// assign an object representing the function to a variable
val f = (x:Int) => x + 1
Since everything is an object in Scala f can now be treated as a reference to Function1[Int,Int] object. For example, we can call toString method inherited from Any, that would have been impossible for a pure function, because functions don't have methods:
f.toString
Or we could define another Function1[Int,Int] object by calling compose method on f and chaining two different functions together:
val f2 = f.compose((x:Int) => x - 1)
Now if we want to actually execute the function, or as mathematician say "apply a function to its arguments" we would call the apply method on the Function1[Int,Int] object:
f2.apply(2)
Writing f.apply(args) every time you want to execute a function represented as an object is the Object-Oriented way, but would add a lot of clutter to the code without adding much additional information and it would be nice to be able to use more standard notation, such as f(args). That's where Scala compiler steps in and whenever we have a reference f to a function object and write f (args) to apply arguments to the represented function the compiler silently expands f (args) to the object method call f.apply (args).
Every function in Scala can be treated as an object and it works the other way too - every object can be treated as a function, provided it has the apply method. Such objects can be used in the function notation:
// we will be able to use this object as a function, as well as an object
object Foo {
var y = 5
def apply (x: Int) = x + y
}
Foo (1) // using Foo object in function notation
There are many usage cases when we would want to treat an object as a function. The most common scenario is a factory pattern. Instead of adding clutter to the code using a factory method we can apply object to a set of arguments to create a new instance of an associated class:
List(1,2,3) // same as List.apply(1,2,3) but less clutter, functional notation
// the way the factory method invocation would have looked
// in other languages with OO notation - needless clutter
List.instanceOf(1,2,3)
So apply method is just a handy way of closing the gap between functions and objects in Scala.
It comes from the idea that you often want to apply something to an object. The more accurate example is the one of factories. When you have a factory, you want to apply parameter to it to create an object.
Scala guys thought that, as it occurs in many situation, it could be nice to have a shortcut to call apply. Thus when you give parameters directly to an object, it's desugared as if you pass these parameters to the apply function of that object:
class MyAdder(x: Int) {
def apply(y: Int) = x + y
}
val adder = new MyAdder(2)
val result = adder(4) // equivalent to x.apply(4)
It's often use in companion object, to provide a nice factory method for a class or a trait, here is an example:
trait A {
val x: Int
def myComplexStrategy: Int
}
object A {
def apply(x: Int): A = new MyA(x)
private class MyA(val x: Int) extends A {
val myComplexStrategy = 42
}
}
From the scala standard library, you might look at how scala.collection.Seq is implemented: Seq is a trait, thus new Seq(1, 2) won't compile but thanks to companion object and apply, you can call Seq(1, 2) and the implementation is chosen by the companion object.
Here is a small example for those who want to peruse quickly
object ApplyExample01 extends App {
class Greeter1(var message: String) {
println("A greeter-1 is being instantiated with message " + message)
}
class Greeter2 {
def apply(message: String) = {
println("A greeter-2 is being instantiated with message " + message)
}
}
val g1: Greeter1 = new Greeter1("hello")
val g2: Greeter2 = new Greeter2()
g2("world")
}
output
A greeter-1 is being instantiated with message hello
A greeter-2 is being instantiated with message world
TLDR for people comming from c++
It's just overloaded operator of ( ) parentheses
So in scala:
class X {
def apply(param1: Int, param2: Int, param3: Int) : Int = {
// Do something
}
}
Is same as this in c++:
class X {
int operator()(int param1, int param2, int param3) {
// do something
}
};
1 - Treat functions as objects.
2 - The apply method is similar to __call __ in Python, which allows you to use an instance of a given class as a function.
The apply method is what turns an object into a function. The desire is to be able to use function syntax, such as:
f(args)
But Scala has both functional and object oriented syntax. One or the other needs to be the base of the language. Scala (for a variety of reasons) chooses object oriented as the base form of the language. That means that any function syntax has to be translated into object oriented syntax.
That is where apply comes in. Any object that has the apply method can be used with the syntax:
f(args)
The scala infrastructure then translates that into
f.apply(args)
f.apply(args) has correct object oriented syntax. Doing this translation would not be possible if the object had no apply method!
In short, having the apply method in an object is what allows Scala to turn the syntax: object(args) into the syntax: object.apply(args). And object.apply(args) is in the form that can then execute.
FYI, this implies that all functions in scala are objects. And it also implies that having the apply method is what makes an object a function!
See the accepted answer for more insight into just how a function is an object, and the tricks that can be played as a result.
To put it crudely,
You can just see it as custom ()operator. If a class X has an apply() method, whenever you call X() you will be calling the apply() method.