I'm trying to create a Scala macro that generates code like:
val x = new com.foo.MyClass()
where com.foo.MyClass is definitely on the classpath at compile time and run time in the project using the macro.
I'm using the following c.Tree to generate the code:
Apply(Select(New(Ident(TermName("com.foo.MyClass"))), termNames.CONSTRUCTOR), List())
Printing the output of the show and showRaw commands indicate that the correct code is generated, however it seems that com.foo.MyClass either isn't on the class path during macro expansion or during compilation immediately after.
I'm seeing the following error generated at the usage point of the macro (the macro impl itself is defined in a separate project):
[ERROR] /src/main/java/foo/MyWhatever.scala:10: not found: value com.foo.MyClass
[ERROR] MyMacros.someMacro(someInput)
[ERROR]
Why is it failing to find this class on the classpath even though it's a Java file in the same project? I tried -Ymacro-debug-verbose and com.foo.MyClass isn't in the output, but a bunch of other Java & Scala classes are. I can't find a pattern to which classes are on the classpath for the Macro expansion.
Thanks for any help!
Okay! I managed to answer my own question. It turns out it works to use c.mirror.staticClass("com.foo.MyClass") to use compile-time reflection to get a class Symbol, then use Quasi-quotes.
My solution:
val classSymbol = c.mirror.staticClass("com.foo.MyClass")
val newClassTree = q"new ${classSymbol.toType}()"
c.Expr { newClassTree } // Success! This compiles and runs
Related
In scala language, implicit resolution is often done in compile-time and sometimes throws obfuscating error information, one famous example of such error is when shapeless Generic throws error information like:
error: could not find implicit value for parameter encoder: CsvEncoder[Foo]
(see https://books.underscore.io/shapeless-guide/shapeless-guide.html for detail)
A solution to this problem is to run implicit resolution algorithm (should be a graph query algorithm internally) in runtime, this has at least 2 benefits:
debugging tools can be used to reproduce the resolution process step-by-step, so even error information & documentations are incomplete it would be easy to spot the error.
in many cases type information can be impossible to be determined in compile-time (e.g. type depending on the control flow). If implicit conversion cannot be delayed to runtime phase, many benefit of defining implicit conversion will be nullified.
So my question is, does this feature exist in Scala 2.x or Dotty? Or is it on the roadmap?
Thanks a lot for your opinion.
You can debug implicits at compile time:
switch on compiler flag -Xlog-implicits
try to resolve implicits manually (maybe specifying type parameters as well) and see compile errors
implicitly[...](...manually...)
use scala.reflect
println(reify { implicitly[...] }.tree)
(or switch on compiler flag -Xprint:typer) in order to see how implicits are resolved
use IDE functionality to show implicits
using macros with compiler internals you can debug implicit resolution
Is there a type-class that checks for existence of at least one implicit of a type?
create an ambiguous low priority implicit
Using the "Prolog in Scala" to find available type class instances
Finding the second matching implicit
shapeless/package.scala#L119-L168 (def cachedImplicitImpl[T](implicit tTag: WeakTypeTag[T]): Tree = ...)
If you're developing a type class don't forget to use annotations #implicitNotFound and #implicitAmbiguous.
You can always postpone compilation of your program till runtime. So instead of program
object App {
def main(args: Array[String]): Unit = {
println("test") // test
}
}
you can have
import scala.reflect.runtime.currentMirror
import scala.reflect.runtime.universe._
import scala.tools.reflect.ToolBox
val toolbox = currentMirror.mkToolBox()
toolbox.eval(q"""
object App {
def main(args: Array[String]): Unit = {
println("test")
}
}
App.main(Array())
""") // test
And instead of
implicitly[Numeric[Int]]
you can have
toolbox.compile(q"""
implicitly[Numeric[Int]]
""")
or
toolbox.inferImplicitValue(
toolbox.typecheck(tq"Numeric[Int]", mode = toolbox.TYPEmode).tpe,
silent = false
)
But it's too optimistic to think that postponing program compilation till runtime you'll be able to debug implicits easier at runtime rather than at compile time. Actually postponing program compilation till runtime you add one level of indirection more i.e. make things harder to debug.
I have an interface:
trait MyInterface {
def doSomething(usefulName : Int) : Unit
}
I have a macro that iterates over the methods of the interface and does stuff with the method names and parameters. I access the method names by doing something like this:
val tpe = typeOf[MyInterface]
// Get lists of parameter names for each method
val listOfParamLists = tpe.decls
.filter(_.isMethod)
.map(_.asMethod.paramLists.head.map(sym => sym.asTerm.name))
If I print out the names for doSomething's parameters, usefulName has become x$1. Why is this happening and is there a way to preserve the original parameter names?
I am using scala version 2.11.8, macros paradise version 2.1.0, and the blackbox context.
The interface is actually java source in a separate sbt project that I control. I have tried compiling with:
javacOptions in (Compile, compile) ++= Seq("-target", "1.8", "-source", "1.8", "-parameters")
The parameters flag is supposed to preserve the names, but I still get the same result as before.
This has nothing to do with macros and everything to do with Scala's runtime reflection system. In a nutshell, Java 8 and Scala 2.11 both wanted to be able to look up parameter names and each implemented their reflection system to do it.
This works just fine if everything is Scala and you compile it together (duh!). Problems arise when you have a Java class that has to be compiled separately.
Observations and Problem
First thing to notice is that the -parameters flag is only since Java 8, which is about as old as Scala 2.11. So Scala 2.11 is probably not using this feature to lookup method names... Consider the following
MyInterface.java compiled with javac -parameters MyInterface.java
public interface MyInterface {
public int doSomething(int bar);
}
MyTrait.scala compiled with scalac MyTrait.scala
class MyTrait {
def doSomething(bar: Int): Int
}
Then, we can use MethodParameterSpy to inspect the parameter information name that the Java 8 -parameter flag is supposed to give us. Running it on the Java compiled interface, we get (and here I abbreviated some of the output)
public abstract int MyInterface.doSomething(int)
Parameter name: bar
but in the Scala compiled class, we only get
public abstract int MyTrait.doSomething(int)
Parameter name: arg0
Yet, Scala has no problem looking up its own parameter names. That tells us that Scala is actually not relying on this Java 8 feature at all - it constructs its own runtime system for keeping track of parameter names. Then, it comes as no surprise that this doesn't work for classes from Java sources. It generates the names x$1, x$2, ... as placeholders, the same way that Java 8 reflection generates the names arg0, arg1, ... as placeholders when we inspected a compiled Scala trait. (And if we had not passed -parameters, it would have generated those names even for MyInterface.java.)
Solution
The best solution (that works in 2.11) I can come up with to get the parameter names of a Java class is to use Java reflection from Scala. Something like
$ javac -parameters MyInterface.java
$ jar -cf MyInterface.jar MyInterface.class
$ scala -cp MyInterface.jar
scala> :pa
// Entering paste mode (ctrl-D to finish)
import java.lang.reflect._
Class.forName("MyInterface")
.getDeclaredMethods
.map(_.getParameters.map(_.getName))
// Exiting paste mode, now interpreting.
res: Array[Array[String]] = Array(Array(bar))
Of course, this will only work if you have the -parameter flag (else you get arg0).
I should probably also mention that if you don't know if your method was compiled from Java or from Scala, you can always call .isJava (For example: typeOf[MyInterface].decls.filter(_.isMethod).head.isJava) and then branch on that to either your initial solution or what I propose above.
Future
Mercifully, this is all a thing of the past in Scala 2.12. If I am correctly reading this ticket, that means that in 2.12 your code will work for Java classes compiled with -parameter and, my Java reflection hack will also work for Scala classes.
All's well that ends well?
I've made use of a few of scala's built-in type classes, and created a few of my own. However, the biggest issue I have with them at the moment is: how do I find type classes available to me? While most of those that I write are small and simple, it would be nice to know if something already exists that does what I'm about to implement!
So, is there a list, somewhere, of all the type classes or implicit values available in the standard library?
Even better, is it possible to somehow (probably within the REPL) generate a list of the implicit values available in the current scope?
It's a job for a good IDE.
IntellijIDEA 14+
Check out Implicits analyser in Scala Plugin 1.4.x. Example usage:
def myMethod(implicit a: Int) = {
}
implicit val a: Int = 1
myMethod // click the myMethod and press Ctrl+Shift+P, the "Implicit Parameters" is shown
Eclipse
Check out Implicit highlighting.
Scala REPL
You can list implicits like this:
:implicits -v
And investigate their origin like defined here:
import reflect.runtime.universe
val tree = universe.reify(1 to 4).tree
universe.showRaw(tree)
universe.show(tree)
Back when reflection was still incipient, on the days of Scala 2.10.0 milestones, I asked a question about how could I use it to see the trees of code snippets from REPL. The excellent answer went further than I asked, and showed how they can be used to parse and evaluate trees as well, so I went ahead and tried to use that on a little project I had going on today.
Unfortunately, code parsed and evaluated that way doesn't seem to see any REPL definition:
scala> val x = 1
x: Int = 1
scala> import scala.tools.reflect.ToolBox
import scala.tools.reflect.ToolBox
scala> val tb = scala.reflect.runtime.universe.runtimeMirror(
getClass.getClassLoader).mkToolBox()
tb: scala.tools.reflect.ToolBox[reflect.runtime.universe.type] = ...
scala> tb.eval(tb.parse("x"))
scala.tools.reflect.ToolBoxError: reflective compilation has failed:
not found: value x
Is there a way to get it to recognize definitions made on REPL?
Recently I dug into repl, when trying to make it support type macros, so I'm well equipped to explain why it doesn't work. Getting it to work would be the next step :)
I know that you know that every snippet entered into repl gets wrapped into some boilerplate before being compiled. Therefore that x ends up being a field in a nested-nested-nested object in a package with a weird name.
Apparently, repl keeps track of all defined symbols and then injects the necessary imports along with the boilerplate it generates. Therefore subsequent lines can see that x unqualified. To the contrast, toolboxes simply reuse repl's classloader, but don't do anything about the imports, hence the failure.
A workaround would be to somehow get to an object representing a repl, ask it about defined symbols and then generate corresponding imports into the code that you feed to a toolbox. If you file a ticket, I'll try to code up a workaround after the 2.10.1 code freeze madness ends (supposedly, end of this week).
I'm trying to compile Scala files programmatically by using an instance of Global.Run:
val settings = new Settings
val reporter = new ConsoleReporter(settings)
val compiler = new Global(settings, reporter)
val run = new compiler.Run // MissingRequirementError
run compile List(path)
Unfortunately I get a MissingRequirementError saying:
object scala.runtime in compiler mirror not found
So my question is how can I compile a file programmatically by using the Run class, or what am I doing wrong here?
I tried to figure out whether I could change the settings in order to get it work. Actually I need a list of classes that are in the Scala file at path, not necessarily a fully runnable output. It would therefore be fine if symbols remained unresolved (if I could run a subset of the compiler phases).
I also at Writing Scala Compiler Plugins, but if I can run it by instanciating a Compiler Run object, I'd prefer this solution. I also stumbled across Is the Scala compiler reentrant? (similar code, different question), which makes me think it might work the way I'm thinking of.
Edit 1: Added Scala JARs to the toolcp (just sample code with absolute path!)
According to a comment I adapted scalac.bat's classpath population script to my Scala code:
// scalac.bat
// if "%_TOOL_CLASSPATH%"=="" (
// for %%f in ("!_SCALA_HOME!\lib\*") do call :add_cpath "%%f"
// for /d %%f in ("!_SCALA_HOME!\lib\*") do call :add_cpath "%%f"
// )
new File("C:\\Program Files\\scala\\lib").listFiles.foreach(f => {
settings.classpath.append(f.getAbsolutePath)
settings.toolcp.append(f.getAbsolutePath)
})
I got it running, by using bootclasspath instead of toolcp (thanks to pedrofurla's hint):
val settings = new Settings
new File("C:\\Program Files\\scala\\lib").listFiles.foreach(f => {
settings.classpath.append(f.getAbsolutePath)
settings.bootclasspath.append(f.getAbsolutePath)
})
private val reporter = new ConsoleReporter(settings)
private val compiler = new Global(settings, reporter)
val run = new compiler.Run
run compile List(path)
The compiler attempts to compile the files now. However, this seems not to be exactly what scalac.bat does. It starts it with -cp, which is the normal classpath, whereas bootclasspath is passed with -bootclasspath on the console, as visible in StandardScalaSettings trait:
val bootclasspath = PathSetting ("-bootclasspath", "Override location of bootstrap class files.", Defaults.scalaBootClassPath)