I'm trying to get started with writing a compiler plugin for scala 3. At this stage, it's primarily based on https://github.com/liufengyun/scala3-plugin-example/blob/master/plugin/src/main/scala/Phases.scala (and the accompanying youtube video explaining how it works).
It's been an interesting process so far, and I'm getting a bit of a feel for some aspects of the compiler.
As a first step, I'm simply trying to wrap a method body into a block, print whatever the returned object was going to be, and then return the object.
This differs from the original plugin mainly in that there was a single side-effecting method call added to each method - this is also assigning a local variable, (which I think is probably the cause of the problem), and moving the method body into a block.
I've produced as minimal of a working example as I could in a fork here: https://github.com/robmwalsh/scala3-plugin-example
The plugin compiles fine, seems to run as part of compilation as expected, and then blows up at runtime. I'm not entirely sure if this is me doing something wrong (not unlikely) or a bug in the compiler (less likely, but a distinct possibility!).
Can anybody please shed some light on why this isn't working? I don't know what flags should be set when creating a new Symbol, so that's one possibility, but there's heaps of stuff that sorta seemed to work so I rolled with it.
Here's where I'm at (the interesting bits):
...
override def prepareForUnit(tree: Tree)(using ctx: Context): Context =
//find the printLn method
val predef = requiredModule("scala.Predef")
printlnSym = predef.requiredMethod("println", List(defn.AnyType))
ctx
override def transformDefDef(tree: DefDef)(using ctx: Context): Tree =
val sym = tree.symbol
// ignore abstract and synthetic methods
if tree.rhs.isEmpty|| sym.isOneOf(Synthetic | Deferred | Private | Accessor)
then return tree
try {
println("\n\n\n\n")
println("========================== tree ==========================")
println(tree.show)
// val body = {tree.rhs}
val body = ValDef(
newSymbol(
tree.symbol, termName("body"), tree.symbol.flags, tree.rhs.tpe),
Block(Nil, tree.rhs)
)
// println(body)
val bodyRef = ref(body.symbol)
val printRes = ref(printlnSym).appliedTo(bodyRef)
// shove it all together in a block
val rhs1 = tpd.Block(body :: printRes :: Nil, bodyRef)
//replace RHS with new
val newDefDef = cpy.DefDef(tree)(rhs = rhs1)
println("====================== transformed ======================")
println(newDefDef.show)
newDefDef
} catch {
case e =>
println("====================== error ===========================")
println(e)
println(e.printStackTrace)
tree
}
...
test program for compiler plugin
object Test extends App:
def foo: String = "forty two"
def bar(x: String): Int = x.length
def baz(x: String, y: Int): String = x + y
baz(foo, bar(foo))
output during compile using plugin (exactly what I wanted! I got very excited at this point)
========================== tree ==========================
def foo: String = "forty two"
====================== transformed ======================
def foo: String =
{
val body: ("forty two" : String) =
{
"forty two"
}
println(body)
body
}
========================== tree ==========================
def bar(x: String): Int = x.length()
====================== transformed ======================
def bar(x: String): Int =
{
val body: Int =
{
x.length()
}
println(body)
body
}
========================== tree ==========================
def baz(x: String, y: Int): String = x.+(y)
====================== transformed ======================
def baz(x: String, y: Int): String =
{
val body: String =
{
x.+(y)
}
println(body)
body
}
output during runtime :'( (this changes depending on the code it's running on, but always the same theme)
Exception in thread "main" java.lang.VerifyError: Bad local variable type
Exception Details:
Location:
testing/Test$.body$2()I #0: aload_1
Reason:
Type top (current frame, locals[1]) is not assignable to reference type
Current Frame:
bci: #0
flags: { }
locals: { 'testing/Test$' }
stack: { }
Bytecode:
0000000: 2bb6 007d ac
at testing.Test.main(Example.scala)
Edit: I'm using scala 3.1.2
I was using the existing flags when creating my new symbol. Instead, I needed to use the Local flag, which I suppose makes sense.
Related
In JS, I can access the function arguments through the arguments object.
I want to do the same thing somehow in ScalaJS, because I want to do some logging for functions and what parameters they got.
Is it possible?
Hmm -- interesting question. I honestly don't know if there is a way to access arguments per se. I would probably address the desire for logging the function arguments by using the sourcecode library, which is designed for that sort of thing, but I'll admit that I haven't tried that from Scala.js yet...
You cannot directly access the arguments object in Scala.js. However, you can export a method with varargs and it will work as expected in JavaScript:
object Logger {
#JSExportTopLevel("log")
def log(xs: js.Any*): Unit = {
xs.foreach(println)
}
}
This defines and exports log to the top level scope. In the JavaScript code, you can now call:
log(1, {}, {a: 1}, "foo");
As #justin-du-coeur suggested, you can use sourcecode for this. For example:
object Test extends js.JSApp {
def main(): Unit = {
a(1, "a")
b()
c("foo", "bar", "baz")
}
def trace()(implicit name: sourcecode.Name, args: sourcecode.Args): Unit = {
def makeArgList(as: Seq[sourcecode.Text[_]]): String =
as.map(a => f"${a.source} = ${a.value}").mkString("(", ", ", ")")
val argStr = args.value.map(makeArgList).mkString("")
println(f"${name.value}$argStr")
}
def a(arg1: Int, arg2: String): Unit = {
trace()
}
def b(): Unit = {
trace()
}
def c(x: String*): Unit = {
trace()
}
}
The output is as follows:
[info] Running Test
a(arg1 = 1, arg2 = a)
b()
c(x = WrappedArray(foo, bar, baz))
As you can see, trace can capture everything it needs from the context, so the result is even more boilerplate free than any JS solution I can think of.
Question
Would like to get assistance to understand the cause of the error. The original is from Coursera Scala Design Functional Random Generators.
Task
With the factories for random int and random boolean, trying to implement a random tree factory.
trait Factory[+T] {
self => // alias of 'this'
def generate: T
def map[S](f: T => S): Factory[S] = new Factory[S] {
def generate = f(self.generate)
}
def flatMap[S](f: T => Factory[S]): Factory[S] = new Factory[S] {
def generate = f(self.generate).generate
}
}
val intFactory = new Factory[Int] {
val rand = new java.util.Random
def generate = rand.nextInt()
}
val boolFactory = intFactory.map(i => i > 0)
Problem
The implementation in the 1st block causes the error but if it changed into the 2nd block, it does not. I believe Factory[+T] meant that Factory[Inner] and Factory[Leaf] could be both treated as Factory[Tree].
I have no idea why the same if expression in for block is OK but it is not OK in yield block. I appreciate explanations.
trait Tree
case class Inner(left: Tree, right: Tree) extends Tree
case class Leaf(x: Int) extends Tree
def leafFactory: Factory[Leaf] = intFactory.map(i => new Leaf(i))
def innerFactory: Factory[Inner] = new Factory[Inner] {
def generate = new Inner(treeFactory.generate, treeFactory.generate)
}
def treeFactory: Factory[Tree] = for {
isLeaf <- boolFactory
} yield if (isLeaf) leafFactory else innerFactory
^^^^^^^^^^^ ^^^^^^^^^^^^
type mismatch; found : Factory[Inner] required: Tree
type mismatch; found : Factory[Leaf] required: Tree
However, below works.
def treeFactory: Factory[Tree] = for {
isLeaf <- boolFactory
tree <- if (isLeaf) leafFactory else innerFactory
} yield tree
I have no idea why the same if expression in for block is OK but it is
not OK in yield block
Because they are translated differently by the compiler. The former example is translated into:
boolFactory.flatMap((isLeaf: Boolean) => if (isLeaf) leafFactory else innerFactor)
Which yields the expected Factory[Tree], while the latter is being translated to:
boolFactory.map((isLeaf: Boolean) => if (isLeaf) leafFactory else innerFactory)
Which yields a Factory[Factory[Tree]], not a Factory[Tree], thus not conforming to your method signature. This isn't about covariance, but rather how for comprehension translates these statements differently.
Is it possible to remove some types from the following code:
import util.continuations._
object TrackingTest extends App {
implicit def trackable(x: Int) = new {
def tracked[R] = shift { cf: (Int => (R, Set[Int])) =>
cf(x) match {
case (r, ints) => (r, ints + x)
}
}
}
def track[R](body: => R #cpsParam[(R, Set[Int]), (R, Set[Int])]) = reset {
(body, Set[Int]())
}
val result = track(7.tracked[Int] + 35.tracked[Int])
assert(result == (42, Set(7, 35)))
val differentTypes = track(9.tracked[String].toString)
assert(differentTypes == ("9", Set(9)))
}
track function tracks calls of tracked on Int instances (e.g. 7.tracked).
Is it possible to infer type parameter on tracked implicit, so the following would compile:
track(7.tracked + 35.tracked)
Your question made me think of how continuations can track state. So I adapted that to your case and came up with this:
import util.continuations._
object TrackingTest extends App {
type State = Set[Int]
type ST = State => State
implicit class Tracked(val i: Int) extends AnyVal {
def tracked = shift{ (k: Int=>ST) => (state:State) => k(i)(state + i) }
}
def track[A](thunk: => A#cps[ST]): (A, State) = {
var result: A = null.asInstanceOf[A]
val finalSate = (reset {
result = thunk
(state:State) => state
}).apply(Set[Int]())
(result, finalSate)
}
val result = track(7.tracked + 35.tracked)
assert(result == (42, Set(7, 35)))
val differentTypes = track(9.tracked.toString)
assert(differentTypes == ("9", Set(9)))
}
This is using 2.10.1 but it works fine with 2.9.1 as well provided you replace the 2.10.x implicit value class with:
implicit def tracked(i: Int) = new {
def tracked = shift{ (k: Int=>ST) => (state:State) => k(i)(state + i) }
}
The key change I made is to have tracked not use any type inference, fixing to Int#cps[ST]. The CPS plugin then maps the computation to the right type (like String#cps[ST]) as appropriate. The state is threaded by the continuation returning a State=>State function that takes the current state (the set of ints) and returns the next state. The return type of reset is a function from state to state (of type ST) that will take the initial state and will return the final state.
The final trick is to use a var to capture the result while still keeping the expected type for reset.
While the exact answer to this question can be given only by the authors of the compiler, we can guess it is not possible by giving a look to the continuation plugin source code.
If you look to the source of the continuations you can see this:
val anfPhase = new SelectiveANFTransform() {
val global = SelectiveCPSPlugin.this.global
val runsAfter = List("pickler")
}
val cpsPhase = new SelectiveCPSTransform() {
val global = SelectiveCPSPlugin.this.global
val runsAfter = List("selectiveanf")
}
The anfPhase phase is executed after the pickler phase, and the cpsPhase after selectiveAnf. If you look to SelectiveANFTransform.scala
abstract class SelectiveANFTransform extends PluginComponent with Transform with
TypingTransformers with CPSUtils {
// inherits abstract value `global' and class `Phase' from Transform
import global._ // the global environment
import definitions._ // standard classes and methods
import typer.atOwner // methods to type trees
/** the following two members override abstract members in Transform */
val phaseName: String = "selectiveanf"
If we use scalac -Xshow-phases, we can see the phases during the compilation process:
parser
namer
packageobjects
typer
superaccessors
pickler
refchecks
selectiveanf
liftcode
selectivecps
uncurry
......
As you can see the typer phase is applied before the selectiveAnf and selectiveCps phases. It should be confirmed that type inference occurs in the typer phase, but if this is really the case and it would make sense, it should be now clear why you can't omit the Int type on 7.tracked and 35.tracked.
Now if you are not satisfied yet, you should know that the compiler works by performing a set of transformations on "trees", which you might look, using the following options:
-Xprint: shows your scala code after a certain phase have been executed
-Xprint: -Yshow-trees shows your scala code and the trees after the phase have been executed
-YBrowse: opens a GUI to surf both.
Having a trait
trait Persisted {
def id: Long
}
how do I implement a method that accepts an instance of any case class and returns its copy with the trait mixed in?
The signature of the method looks like:
def toPersisted[T](instance: T, id: Long): T with Persisted
This can be done with macros (that are officially a part of Scala since 2.10.0-M3). Here's a gist example of what you are looking for.
1) My macro generates a local class that inherits from the provided case class and Persisted, much like new T with Persisted would do. Then it caches its argument (to prevent multiple evaluations) and creates an instance of the created class.
2) How did I know what trees to generate? I have a simple app, parse.exe that prints the AST that results from parsing input code. So I just invoked parse class Person$Persisted1(first: String, last: String) extends Person(first, last) with Persisted, noted the output and reproduced it in my macro. parse.exe is a wrapper for scalac -Xprint:parser -Yshow-trees -Ystop-after:parser. There are different ways to explore ASTs, read more in "Metaprogramming in Scala 2.10".
3) Macro expansions can be sanity-checked if you provide -Ymacro-debug-lite as an argument to scalac. In that case all expansions will be printed out, and you'll be able to detect codegen errors faster.
edit. Updated the example for 2.10.0-M7
It is not possible to achieve what you want using vanilla scala. The problem is that the mixins like the following:
scala> class Foo
defined class Foo
scala> trait Bar
defined trait Bar
scala> val fooWithBar = new Foo with Bar
fooWithBar: Foo with Bar = $anon$1#10ef717
create a Foo with Bar mixed in, but it is not done at runtime. The compiler simply generates a new anonymous class:
scala> fooWithBar.getClass
res3: java.lang.Class[_ <: Foo] = class $anon$1
See Dynamic mixin in Scala - is it possible? for more info.
What you are trying to do is known as record concatenation, something that Scala's type system does not support. (Fwiw, there exist type systems - such as this and this - that provide this feature.)
I think type classes might fit your use case, but I cannot tell for sure as the question doesn't provide sufficient information on what problem you are trying to solve.
Update
You can find an up to date working solution, which utilizes a Toolboxes API of Scala 2.10.0-RC1 as part of SORM project.
The following solution is based on the Scala 2.10.0-M3 reflection API and Scala Interpreter. It dynamically creates and caches classes inheriting from the original case classes with the trait mixed in. Thanks to caching at maximum this solution should dynamically create only one class for each original case class and reuse it later.
Since the new reflection API isn't that much disclosed nor is it stable and there are no tutorials on it yet this solution may involve some stupid repitative actions and quirks.
The following code was tested with Scala 2.10.0-M3.
1. Persisted.scala
The trait to be mixed in. Please note that I've changed it a bit due to updates in my program
trait Persisted {
def key: String
}
2. PersistedEnabler.scala
The actual worker object
import tools.nsc.interpreter.IMain
import tools.nsc._
import reflect.mirror._
object PersistedEnabler {
def toPersisted[T <: AnyRef](instance: T, key: String)
(implicit instanceTag: TypeTag[T]): T with Persisted = {
val args = {
val valuesMap = propertyValuesMap(instance)
key ::
methodParams(constructors(instanceTag.tpe).head.typeSignature)
.map(_.name.decoded.trim)
.map(valuesMap(_))
}
persistedClass(instanceTag)
.getConstructors.head
.newInstance(args.asInstanceOf[List[Object]]: _*)
.asInstanceOf[T with Persisted]
}
private val persistedClassCache =
collection.mutable.Map[TypeTag[_], Class[_]]()
private def persistedClass[T](tag: TypeTag[T]): Class[T with Persisted] = {
if (persistedClassCache.contains(tag))
persistedClassCache(tag).asInstanceOf[Class[T with Persisted]]
else {
val name = generateName()
val code = {
val sourceParams =
methodParams(constructors(tag.tpe).head.typeSignature)
val newParamsList = {
def paramDeclaration(s: Symbol): String =
s.name.decoded + ": " + s.typeSignature.toString
"val key: String" :: sourceParams.map(paramDeclaration) mkString ", "
}
val sourceParamsList =
sourceParams.map(_.name.decoded).mkString(", ")
val copyMethodParamsList =
sourceParams.map(s => s.name.decoded + ": " + s.typeSignature.toString + " = " + s.name.decoded).mkString(", ")
val copyInstantiationParamsList =
"key" :: sourceParams.map(_.name.decoded) mkString ", "
"""
class """ + name + """(""" + newParamsList + """)
extends """ + tag.sym.fullName + """(""" + sourceParamsList + """)
with """ + typeTag[Persisted].sym.fullName + """ {
override def copy(""" + copyMethodParamsList + """) =
new """ + name + """(""" + copyInstantiationParamsList + """)
}
"""
}
interpreter.compileString(code)
val c =
interpreter.classLoader.findClass(name)
.asInstanceOf[Class[T with Persisted]]
interpreter.reset()
persistedClassCache(tag) = c
c
}
}
private lazy val interpreter = {
val settings = new Settings()
settings.usejavacp.value = true
new IMain(settings, new NewLinePrintWriter(new ConsoleWriter, true))
}
private var generateNameCounter = 0l
private def generateName() = synchronized {
generateNameCounter += 1
"PersistedAnonymous" + generateNameCounter.toString
}
// REFLECTION HELPERS
private def propertyNames(t: Type) =
t.members.filter(m => !m.isMethod && m.isTerm).map(_.name.decoded.trim)
private def propertyValuesMap[T <: AnyRef](instance: T) = {
val t = typeOfInstance(instance)
propertyNames(t)
.map(n => n -> invoke(instance, t.member(newTermName(n)))())
.toMap
}
private type MethodType = {def params: List[Symbol]; def resultType: Type}
private def methodParams(t: Type): List[Symbol] =
t.asInstanceOf[MethodType].params
private def methodResultType(t: Type): Type =
t.asInstanceOf[MethodType].resultType
private def constructors(t: Type): Iterable[Symbol] =
t.members.filter(_.kind == "constructor")
private def fullyQualifiedName(s: Symbol): String = {
def symbolsTree(s: Symbol): List[Symbol] =
if (s.enclosingTopLevelClass != s)
s :: symbolsTree(s.enclosingTopLevelClass)
else if (s.enclosingPackageClass != s)
s :: symbolsTree(s.enclosingPackageClass)
else
Nil
symbolsTree(s)
.reverseMap(_.name.decoded)
.drop(1)
.mkString(".")
}
}
3. Sandbox.scala
The test app
import PersistedEnabler._
object Sandbox extends App {
case class Artist(name: String, genres: Set[Genre])
case class Genre(name: String)
val artist = Artist("Nirvana", Set(Genre("rock"), Genre("grunge")))
val persisted = toPersisted(artist, "some-key")
assert(persisted.isInstanceOf[Persisted])
assert(persisted.isInstanceOf[Artist])
assert(persisted.key == "some-key")
assert(persisted.name == "Nirvana")
assert(persisted == artist) // an interesting and useful effect
val copy = persisted.copy(name = "Puddle of Mudd")
assert(copy.isInstanceOf[Persisted])
assert(copy.isInstanceOf[Artist])
// the only problem: compiler thinks that `copy` does not implement `Persisted`, so to access `key` we have to specify it manually:
assert(copy.asInstanceOf[Artist with Persisted].key == "some-key")
assert(copy.name == "Puddle of Mudd")
assert(copy != persisted)
}
While it's not possible to compose an object AFTER it's creation, you can have very wide tests to determine if the object is of a specific composition using type aliases and definition structs:
type Persisted = { def id: Long }
class Person {
def id: Long = 5
def name = "dude"
}
def persist(obj: Persisted) = {
obj.id
}
persist(new Person)
Any object with a def id:Long will qualify as Persisted.
Achieving what I THINK you are trying to do is possible with implicit conversions:
object Persistable {
type Compatible = { def id: Long }
implicit def obj2persistable(obj: Compatible) = new Persistable(obj)
}
class Persistable(val obj: Persistable.Compatible) {
def persist() = println("Persisting: " + obj.id)
}
import Persistable.obj2persistable
new Person().persist()
I was wondering if scala had an equivalent to java's #SuppressWarnings that can be applied to a function or whatever to ignore any deprecation warnings[1] that function emits?
1: Relevant warning in my case is: method stop in class Thread is deprecated: see corresponding Javadoc for more information. I am aware of the problems with stop however there are still some cases where due to legacy code we have to use it.
No, and an enhancement request [1] for such a feature was closed as wontfix.
I agree it would be useful. I expect that the Scala core team aren't against the idea, but they have finite resources and many higher priorities.
update: this feature was eventually implemented in scala 2.13.2 release on 2020-04-22, see this answer
[1] https://issues.scala-lang.org/browse/SI-1781
EDIT: You should use #nowarn
There is a simple compiler plugin for this: silencer (a bit shameless plug)
Scala 2.13.2 provides #nowarn annotation developed on the basis of ghik's silencer, for example
import scala.annotation.nowarn
def t = { 0: #nowarn; 1 }
raises no warnings, whilst
def t = { 0; 1 }
gives
warning: a pure expression does nothing in statement position; multiline expressions might require enclosing parentheses
def t = { 0; 1 }
^
Here is how to suppress all warnings in sbt:
import sbt._
import Keys._
import KeyRanks.DTask
import xsbti.{Reporter, Problem, Position, Severity}
private lazy val compilerReporter = TaskKey[xsbti.Reporter](
"compilerReporter",
"Experimental hook to listen (or send) compilation failure messages.",
DTask
)
val ignoreWarnings = Seq(
compilerReporter in (Compile, compile) :=
new xsbti.Reporter {
private val buffer = collection.mutable.ArrayBuffer.empty[Problem]
def reset(): Unit = buffer.clear()
def hasErrors: Boolean = buffer.exists(_.severity == Severity.Error)
def hasWarnings: Boolean = buffer.exists(_.severity == Severity.Warn)
def printSummary(): Unit = {
print("\033c")
if (problems.nonEmpty) {
problems.foreach{ p =>
println("=====================================================")
println(p.position)
println(p.message)
println()
println()
}
}
}
def problems: Array[Problem] = buffer.toArray
def log(problem: Problem): Unit = {
if (problem.severity == Severity.Error) {
buffer.append(problem)
}
}
def log(pos: Position, msg: String, sev: Severity): Unit = {
log(new Problem {
def category: String = "foo"
def severity: Severity = sev
def message: String = msg
def position: Position = pos
})
}
def comment(pos: xsbti.Position, msg: String): Unit = ()
}
)