What can cause "Unexpected tree in genLoad" in a macro expansion? - scala

I was trying to produce a small macro to isolate another problem that I was having and started running into this compile-time error.
Error:scalac:
Unexpected tree in genLoad: test.MacroTest$Baz.type/class scala.reflect.internal.Trees$TypeTree at: source-/Users/jpatterson/test/src/test/scala/test/MacroTest.scala,line-5,offset=114
while compiling: /Users/jpatterson/test/src/test/scala/test/MacroTest.scala
during phase: jvm
library version: version 2.13.0-RC1
compiler version: version 2.13.0-RC1
reconstructed args: -deprecation -Vimplicits -language:higherKinds -language:implicitConversions -language:postfixOps -classpath ... (cut)
last tree to typer: Literal(Constant(test.MacroTest.MacroTest$Baz.type))
tree position: line 4 of /Users/jpatterson/test/src/test/scala/test/MacroTest.scala
tree tpe: Class(classOf[test.MacroTest$Baz$])
symbol: null
call site: constructor MacroTest$$anon$1 in package test
== Source file context for tree position ==
1 package test
2
3 object MacroTest {
4 case class Baz(x: Int, y: Int)
5 implicit def bazRead: Read[Baz] = Read.readFor[Baz]
6
7 def main(args: Array[String]): Unit = {
I started working with scala 2.12.8. I tried switching to 2.13.0-RC1 just to see if it was something that was already fixed. It fails the same with both versions of scala.
The macro code:
package test
import scala.language.experimental.macros
import scala.reflect.macros.whitebox.Context
trait Read[A] {
def read(in: String): A
}
object Read {
implicit def intRead = new Read[Int] {
override def read(in: String): Int = in.toInt
}
def CaseClassReadImpl[A: c.WeakTypeTag](c: Context): c.Expr[Read[A]] = {
import c.universe._
val aType = weakTypeOf[A]
val params = aType.decls.collect {
case m: MethodSymbol if m.isCaseAccessor => m
}.toList
val paramList = params.map(param => q"Read.read[${param.typeSignature}](in)")
val src = q"""
new Read[$aType] {
def read(in: String) = ${aType.companion}.apply(..$paramList)
}
"""
println(src)
c.Expr[Read[A]](src)
}
def readFor[A]: Read[A] = macro CaseClassReadImpl[A]
def read[A](in: String)(implicit A: Read[A]): A = A.read(in)
}
The code that exercises it:
package test
object MacroTest {
case class Baz(x: Int, y: Int)
implicit def bazRead: Read[Baz] = Read.readFor[Baz]
def main(args: Array[String]): Unit = {
println(Read.read[Baz]("4"))
}
}
Compiling the second block causes the error above.
I was expecting this to compile correctly. I put that println into the macro definition so that I could just grab the code and try to compile that. When I add that to the second block, it compiles fine. I can even replace bazRead's value with it and everything works as expected: it prints out Baz(4,4).

Regarding your macro, you're trying to splice a type (aType.companion) into a position where a term is expected (a tpe: Type is transformed into TypeTree(tpe)).
Try to replace ${aType.companion} with ${aType.typeSymbol.companion}.
For deriving type classes it's better to use Shapeless, Magnolia or Scalaz-deriving than raw macros.
For example in Shapeless Read can be derived as follows
import shapeless.{Generic, HList, HNil, ::}
trait Read[A] {
def read(in: String): A
}
object Read {
implicit def intRead: Read[Int] = _.toInt
implicit def hNilRead: Read[HNil] = _ => HNil
implicit def hConsRead[H, T <: HList](implicit r: Read[H], r1: Read[T]): Read[H :: T] =
in => r.read(in) :: r1.read(in)
implicit def caseClassRead[A, L <: HList](implicit gen: Generic.Aux[A, L], r: Read[L]): Read[A] =
in => gen.from(r.read(in))
def read[A](in: String)(implicit A: Read[A]): A = A.read(in)
}
case class Baz(x: Int, y: Int)
Read.read[Baz]("123") // Baz(123,123)

Related

Generic field accessor powered by Shapeless

I'm trying to implement a convenient generic field accessor based on LabelledGeneric.
The usage should look like:
case class Foo(aha: String, uhu: Double, ehe: Int)
case class Bar(uhu: Double, ahu: Boolean)
val foo: Foo = ???
val bar: Bar = ???
val uhuGenField = new GenField('uhu)
val uhuFooAccess = uhuGenField.from[Foo]
val uhuBarAccess = uhuGenField.from[Bar]
def someFunWithUhu[X](xs: Seq[X], access: uhuGenField.Access[X]) = ???
I spent some time trying to figure out how to achieve such behaviour.
Eventually I came up with this approach:
import shapeless._
import shapeless.ops.record.Selector
final class GenField[V](val fieldName: Symbol) {
val fieldWitness = Witness(fieldName)
type FieldNameType = fieldWitness.T
trait Access[C] {
def get(c: C): V
}
def from[C](implicit lg2hl: LGtoHL[C]): Access[C] = new Access[C] {
override def get(c: C): V = {
val labelledGeneric = lg2hl.labelledGeneric
val selector = Selector.mkSelector[labelledGeneric.Repr, FieldNameType, V]
selector(labelledGeneric.to(c))
}
}
}
// I need something like this to enable syntax like
// genField.from[DesiredClass]
// i.e. to "latch" Repr inside a single instance
// and to don't pass it explicitly to `from` method.
sealed trait LGtoHL[A] {
type Repr <: HList
val labelledGeneric: LabelledGeneric.Aux[A, Repr]
}
object LGtoHL {
implicit def mkLGtoHL[A, ARepr <: HList](implicit lg: LabelledGeneric.Aux[A, ARepr]): LGtoHL[A] = {
new LGtoHL[A] {
override type Repr = ARepr
override val labelledGeneric: LabelledGeneric.Aux[A, Repr] = lg
}
}
}
From my prospective this solution should be OK, but it still doesn't work.
The compilation fails with the following error message:
Error:(17, 41) lg2hl.Repr is not an HList type
val selector = Selector.mkSelector[labelledGeneric.Repr, FieldNameType, V]
Why does it complain lg2hl.Repr is not an HList type?
Repr is explicitly defined in LGtoHL as type Repr <: HList.
What is wrong with my code?
Very appreciate your help.
Why are lenses not enough?
import shapeless.{Lens, lens}
case class Foo(aha: String, uhu: Double, ehe: Int)
case class Bar(uhu: Double, ahu: Boolean)
val foo: Foo = Foo("a", 1.0, 2)
val bar: Bar = Bar(3.0, true)
val fooUhu: Lens[Foo, Double] = lens[Foo] >> 'uhu
val barUhu: Lens[Bar, Double] = lens[Bar] >> 'uhu
fooUhu.get(foo) // 1.0
barUhu.get(bar) // 3.0
The error message
lg2hl.Repr is not an HList type
comes from here: https://github.com/milessabin/shapeless/blob/master/core/src/main/scala/shapeless/generic.scala#L511
u.baseType(HConsSym) is now NoType.
I guess GenField[V](val fieldName: Symbol) will not work since fieldName in Witness(fieldName) must be known at compile time. For example
lens[Foo] >> 'uhu
works but
val uhu: Witness.`'uhu`.T = 'uhu.narrow
lens[Foo] >> uhu
doesn't. This is the reason why lenses, Witness, LabelledGeneric are implemented via macros.

How to use ClassTag in scala macros implemented for type

I wrote a macros, that reads class fields:
import scala.language.experimental.macros
import scala.reflect.macros.whitebox
object ArrayLikeFields {
def extract[T]: Set[String] = macro extractImpl[T]
def extractImpl[T: c.WeakTypeTag](c: whitebox.Context): c.Expr[Set[String]] = {
import c.universe._
val tree = weakTypeOf[T].decls
.collectFirst {
case m: MethodSymbol if m.isPrimaryConstructor => m
}
.map(y => y.paramLists.headOption.getOrElse(Seq.empty))
.getOrElse(Seq.empty)
.map(s => q"${s.name.decodedName.toString}")
c.Expr[Set[String]] {
q"""Set(..$tree)"""
}
}
}
I'm able to compile and run it for concrete type:
object Main extends App {
case class Person(name:String)
val res: Set[String] = ArrayLikeFields.extract[Person]
}
But i want use it with generic types like that:
object Lib {
implicit class SomeImplicit(s: String) {
def toOrgJson[T]: JSONObject = {
val arrayLikeFields: Set[String] = ArrayLikeFields.extract[T]
//some code, that uses fields, etc
null
}
}
}
Compilation error:
Error:(14, 65) type mismatch; found :
scala.collection.immutable.Set[Nothing] required: Set[String] Note:
Nothing <: String, but trait Set is invariant in type A. You may wish
to investigate a wildcard type such as _ <: String. (SLS 3.2.10)
val arrayLikeFields: Set[String] = ArrayLikeFields.extract[T]
I can't understund that. How can I solve my problem?
upd
I read scala 2.10.2 calling a 'macro method' with generic type not work about materialisation, but i have no instance of class
Try approach with materializing a type class like in 1
object Main extends App {
case class Person(name:String)
val res: Set[String] = ArrayLikeFields.extract[Person] //Set(name)
import Lib._
"abc".toOrgJson[Person] // prints Set(name)
}
object Lib {
implicit class SomeImplicit(s: String) {
def toOrgJson[T: ArrayLikeFields.Extract]: JSONObject = {
val arrayLikeFields: Set[String] = ArrayLikeFields.extract[T]
//some code, that uses fields, etc
println(arrayLikeFields) //added
null
}
}
}
import scala.language.experimental.macros
import scala.reflect.macros.whitebox
object ArrayLikeFields {
def extract[T](implicit extr: Extract[T]): Set[String] = extr()
trait Extract[T] {
def apply(): Set[String]
}
object Extract {
implicit def materializeExtract[T]: Extract[T] = macro materializeExtractImpl[T]
def materializeExtractImpl[T: c.WeakTypeTag](c: whitebox.Context): c.Expr[Extract[T]] = {
import c.universe._
val tree = weakTypeOf[T].decls
.collectFirst {
case m: MethodSymbol if m.isPrimaryConstructor => m
}
.map(y => y.paramLists.headOption.getOrElse(Seq.empty))
.getOrElse(Seq.empty)
.map(s => q"${s.name.decodedName.toString}")
c.Expr[Extract[T]] {
q"""new ArrayLikeFields.Extract[${weakTypeOf[T]}] {
override def apply(): _root_.scala.collection.immutable.Set[_root_.java.lang.String] =
_root_.scala.collection.immutable.Set(..$tree)
}"""
}
}
}
}
Actually, I don't think you need whitebox macros here, blackbox ones should be enough. So you can replace (c: whitebox.Context) with (c: blackbox.Context).
By the way, the same problem can be solved with Shapeless rather than macros (macros work in Shapeless under the hood)
object Main extends App {
case class Person(name:String)
val res: Set[String] = ArrayLikeFields.extract[Person] //Set(name)
}
object ArrayLikeFields {
def extract[T: Extract]: Set[String] = implicitly[Extract[T]].apply()
trait Extract[T] {
def apply(): Set[String]
}
object Extract {
def instance[T](strs: Set[String]): Extract[T] = () => strs
implicit def genericExtract[T, Repr <: HList](implicit
labelledGeneric: LabelledGeneric.Aux[T, Repr],
extract: Extract[Repr]
): Extract[T] = instance(extract())
implicit def hconsExtract[K <: Symbol, V, T <: HList](implicit
extract: Extract[T],
witness: Witness.Aux[K]
): Extract[FieldType[K, V] :: T] =
instance(extract() + witness.value.name)
implicit val hnilExtract: Extract[HNil] = instance(Set())
}
}
The answer on the linked question, scala 2.10.2 calling a 'macro method' with generic type not work , also applies here.
You are trying to solve a run-time problem with a compile-time macro, which is not possible.
The called method toOrgJson[T] cannot know the concrete type that T represents at compile time, but only gets that information at run-time. Therefore, you will not be able to do any concrete operations on T (such as listing its fields) at compile-time, only at run-time.
You can implement an operation like ArrayLikeFields.extract[T] at run-time using Reflection, see e.g. Get field names list from case class
I don't have a very solid understanding of Macros, but it seems that the compiler does not understand that the return type of the macro function is Set[String].
The following trick worked for me in scala 2.12.7
def toOrgJson[T]: JSONObject = {
val arrayLikeFields: Set[String] = ArrayLikeFields.extract[T].map(identity[String])
//some code, that uses fields, etc
null
}
EDIT
Actually to get a non empty Set T needs an upper bound such as T <: Person... and that is not what you wanted...
Leaving the answer here since the code does compile, and it might help someone in the direction of an answer

Mapping of String to Type

I am currently trying to write a method that takes a JSON (what API does not matter here) and validates it. I want the method to look something like this:
def validateJson(json, expectedType: Map[String, Type(?)], allowedVals: Map[String, Seq[expectedType(key)]]): Boolean
The problem is: I do have a method jsonfield.validate[expectedType] but I do not know how to pass an unknown number of useable type parameters associated with strings to a method.
I'd gladly use some runtime reflection if that is possible here, or any advanced feature necessary to make this work easily. Any suggestions appreciated.
PS: I am using Play Framework 2.6.3
Edit:
I am trying to use the passed types like this
val allowed = allowedVals(field) // a Set
// if field contents contained in allowed value set...
if( allowed(field.validate[expectedType(field)].get) ) foo
Maybe you can use varargs in runtime or abstract over arity in compile time or just use an HList:
def foo[L <: HList](l: L) = ???
trait A
trait B
trait C
val a: A = new A {}
val b: B = new B {}
val c: C = new C {}
foo[A :: B :: C :: HNil](a :: b :: c :: HNil)
Sounds like you're looking for dependent type/dependent function/polymorphic function:
import shapeless.Poly1
import shapeless.syntax.singleton._
object expectedTypeAndValue extends Poly1 {
implicit val aCase: Case.Aux["a", Int] = at["a"](_ => 1)
implicit val bCase: Case.Aux["b", Long] = at["b"](_ => 2L)
implicit val cCase: Case.Aux["c", Double] = at["c"](_ => 3.0)
}
def validateJson(json: Json): Boolean = {
val x: Long = expectedTypeAndValue["b"]("b".narrow)
???
}
in Typelevel Scala or
import shapeless.{Poly1, Witness}
import shapeless.syntax.singleton._
object expectedTypeAndValue extends Poly1 {
implicit val aCase: Case.Aux[Witness.`"a"`.T, Int] = at[Witness.`"a"`.T](_ => 1)
implicit val bCase: Case.Aux[Witness.`"b"`.T, Long] = at[Witness.`"b"`.T](_ => 2L)
implicit val cCase: Case.Aux[Witness.`"c"`.T, Double] = at[Witness.`"c"`.T](_ => 3.0)
}
def validateJson(json: Json): Boolean = {
val x: Long = expectedTypeAndValue[Witness.`"b"`.T]("b".narrow)
???
}
in Lightbend Scala (ordinary Scala).
You can create also custom type class:
trait ExpectedTypeAndVals[S <: String] {
type Out
def apply(s: S): Set[Out]
}
object ExpectedTypeAndVals {
type Aux[S <: String, Out0] = ExpectedTypeAndVals[S] {type Out = Out0}
implicit def mkExpectedTypeAndVals[S <: String]: ExpectedTypeAndVals.Aux[S, ???] =
new ExpectedTypeAndVals[S] {
override type Out = ???
override def apply(s: S): Set[Out] = ???
}
}
def allowed[S <: String, Out](json: Json)(implicit
typeAndVals: ExpectedTypeAndVals.Aux[S, Out]
): Boolean = {
val str: S = ???
val set: Set[Out] = typeAndVals(str)
???
}
if(allowed(json)) {
???
}

enable implicit import for runtime type in scala check. "could not find implicit value for parameter"

I'm testing my own home-brewed Monoid classes in scala using the ScalaCheck library and ScalaTest
when attempting to implement DRY tests, I get the implicit error in the title:
Error:(16, 12) could not find implicit value for parameter arbA: org.scalacheck.Arbitrary[A]
forAll { (a: A) =>
^
here is the implementation of intAddition Monoid:
trait Monoid[A] {
def op(a1: A, a2: A): A
def zero: A
}
object Monoid {
...
val intAddition: Monoid[Int] = new Monoid[Int] {
override def op(a1: Int, a2: Int): Int = a1 + a2
override def zero: Int = 0
}
...
}
And the test suite:
import org.fpinscala.monoids.Monoid._
import org.fpinscala.testutils.UnitSpec
import org.scalatest.prop.PropertyChecks
import org.scalacheck.Arbitrary._
import scala.language.implicitConversions
class MonoidSpec extends UnitSpec with PropertyChecks {
def assertIdentityBehaviour[A](M: Monoid[A]): Unit = {
import M._
forAll { (a: A) =>
op(zero, a) should be(a)
op(a, zero) should be(a)
}
}
behavior of "intAdditionMonoid"
it should "obey identity laws" in {
assertIdentityBehaviour(intAddition)
}
}
This code compiles but fails at runtime (runtime type erasure?).
Is what I'm trying to achieve possible in Scala?
This code compiles
It doesn't; the error you give is a compilation error. It should be fixed by adding the implicit parameter it complains about:
def assertIdentityBehaviour[A](M: Monoid[A])(implicit arbA: Arbitrary[A]) = ...
// or equivalently, def assertIdentityBehaviour[A: Arbitrary](M: Monoid[A]) = ...
You are calling assertIdentityBehaviour only with A for which the parameter is available, but the error is in its definition.

Scala: implicits, subclassing and member types

Lets say we want to use type classes to implement pretty printing:
trait Printer[T] {def print(t: T)}
with default implementation for ints:
implicit object IntPrinter extends Printer[Int] {
override def print(i : Int): Unit = println(i)
}
Our specific types we want to print are:
trait Foo {
type K
val k: K
}
class IntFoo extends Foo {
override type K = Int
override val k = 123
}
cool. Now I want to build printers for all Foos with printable Ks
implicit def fooPrinter[FP <: Foo](implicit ev: Printer[FP#K]): Printer[FP] =
new Printer[FP] {
override def print(f: FP): Unit = {
Predef.print("Foo: ")
ev.print(f.k)
}
}
lets check that implicits are resolved:
def main(args: Array[String]) {
implicitly[Printer[Int]]
implicitly[Printer[IntFoo]]
}
scalac 2.11.2 says:
diverging implicit expansion for type Sandbox.Printer[Int]
starting with method fooPrinter in object Sandbox
implicitly[Printer[Int]]
whaat?
OK, lets rewrite fooPrinter:
implicit def fooPrinter[KP, FP <: Foo {type K = KP}](implicit ev: Printer[KP]) =
new Printer[FP] {
override def print(f: FP): Unit = {
Predef.print("Foo: ")
ev.print(f.k)
}
}
this works in 2.11, but what's the problem with the first approach?
Unfortunately we're on 2.10, and second solution still doesn't work. It compiles until we add one more sime printer like
implicit object StringPrinter extends Printer[String] {
override def print(s : String): Unit = println(s)
}
and it mysteriously breaks Printer[IntFoo] implicit:
could not find implicit value for parameter e:
Sandbox.Printer[Sandbox.IntFoo]
compiler bugs?
Order of implicit declarations matters. In your source code reorder original code from
implicit object IntPrinter ...
...
implicit def fooPrinter ...
to
implicit def fooPrinter ...
...
implicit object IntPrinter ...