Workaround for Scala macro annotation bug - scala

I have a macro annotation that I use to inject implicit type class to a companion method.
#MyMacro case class MyClass[T](a: String, b: Int, t: T)
Most of the time it work as expected, but it breaks when I use type constraint notation:
#MyMacro case class MyClass[T: TypeClass](a: String, b: Int, t: T)
// private[this] not allowed for case class parameters
This error was described on SO and reported as a bug.
Thing is: macros (v1) are no longer maintained, so I cannot expect that this will be fixed.
So what I wanted to know is: can I fix this myself within a macro? Is this change done to AST in a way that I could somehow undo it? I would like to try repairing it within a macro instead of forcing all users to rewrite their code to ...(implicit tc: TypeClass[T]).

class AnnotationType() extends scala.annotation.StaticAnnotation {
def macroTransform(annottees: Any*): Any = macro AnnotationTypeImpl.impl
}
class AnnotationTypeImpl(val c: blackbox.Context) {
import c.universe._
def impl(annottees: Tree*): Tree = {
val tree = annottees.head.asInstanceOf[ClassDef]
val newTree = tree match {
case ClassDef(mods, name, tparams, impl#Template(parents, self, body)) =>
val newBody = body.map {
case ValDef(mods, name, tpt, rhs) =>
// look here
// the flag of `private[this]` is Flag.PRIVATE | Flag.LOCAL
// the flag of `private` is Flag.PRIVATE
// drop Flag.LOCAL in Modifiers.flags , it will change `private[this]` to `private`
val newMods =
if(mods.hasFlag(Flag.IMPLICIT))
mods.asInstanceOf[scala.reflect.internal.Trees#Modifiers].&~(Flag.LOCAL.asInstanceOf[Long]).&~(Flag.CASEACCESSOR.asInstanceOf[Long]).asInstanceOf[Modifiers]
else
mods
ValDef(newMods, name, tpt, rhs)
case e => e
}
ClassDef(mods, name, tparams, Template(parents, self, newBody))
}
println(show(tree))
println(show(newTree))
q"..${newTree +: annottees.tail}"
}
}
// test
#AnnotationType()
case class AnnotationTypeTest[T: Option](a: T){
def option = implicitly[Option[T]]
}
object AnnotationTypeTest {
def main(args: Array[String]): Unit = {
implicit val x = Option(1)
println(AnnotationTypeTest(100))
println(AnnotationTypeTest(100).option)
println(AnnotationTypeTest(100).copy(a =2222))
println(AnnotationTypeTest(100).copy(a =2222)(Some(999)).option)
}
}

Related

Scala - Get string representation of object property name, not value, for comparison

I want to be able to get the string representation of an objects property name, not the properties value, so that I can compare it with a variables value inside a conditional statement.
case class CustomObj(name: T)
case class PropertyObj(property: String)
val custObj = CustomObj("Chris")
val propObj = PropertyObj("name")
if(propObj.property.equals(custObj. /* the property name as a String, so "name", not the value ("Chris"*/)) {
// do something
}
How can I access what is essentially the key of the property on the CustomObj?
Try productElementNames like so
case class CustomObj(name: String)
case class PropertyObj(property: String)
val custObj = CustomObj("Chris")
val propObj = PropertyObj("name")
if (custObj.productElementNames.toList.headOption.contains(propObj.property)) { ... } else { ... }
Addressing the comment, based on Krzysztof, try shapeless solution
import shapeless._
import shapeless.ops.record._
def firstPropertyNameOf[P <: Product, L <: HList](p: P)(implicit
gen: LabelledGeneric.Aux[P, L],
toMap: ToMap[L]): Option[String] = {
toMap(gen.to(p)).map{ case (k: Symbol, _) => k.name }.toList.headOption
}
firstPropertyNameOf(custObj).contains(propObj.property) // res1: Boolean = true
I will assume you don't know the type of custObj at compile time. Then you'll have to use runtime reflection in Scala 2.12.
scala> case class CustomObj(name: String)
defined class CustomObj
scala> val custObj: Any = CustomObj("Chris")
custObj: Any = CustomObj(Chris)
scala> import scala.reflect.runtime.currentMirror
import scala.reflect.runtime.currentMirror
scala> val sym = currentMirror.classSymbol(custObj.getClass)
sym: reflect.runtime.universe.ClassSymbol = class CustomObj
scala> val props = sym.info.members.collect{ case m if m.isMethod && m.asMethod.isCaseAccessor => m.name.toString }
props: Iterable[String] = List(name)
scala> if (props.exists(_ == "name")) println("ok")
ok

How to create a random instance of a case class?

Suppose I've got a few case classes, e.g.:
case class C(c1: Int, c2: Double, c3: Option[String])
case class B(b: Int, cs: Seq[C])
case class A(a: String, bs: Seq[B])
Now I would like to generate a few instances of A with random values for tests.
I am looking for a generic way to do that. I can probably do it with runtime reflection but I prefer a compile-time solution.
def randomInstance[A](a: A): A = ???
How can I do it ? Can it be done with shapeless ?
The easiest way for you to do that would be using ScalaCheck. You do so by defining a Gen[A] for your instances:
import org.scalacheck.Gen
final case class C(c1: Int, c2: Double, c3: Option[String])
object C {
val cGen: Gen[C] = for {
c1 <- Gen.posNum[Int]
c2 <- Gen.posNum[Double]
c3 <- Gen.option(Gen.oneOf("foo", "bar", "hello"))
} yield C(c1, c2, c3)
}
And you consume it:
object F {
def main(args: Array[String]): Unit = {
val randomC: C = C.cGen.sample.get
}
}
On top of that, you can add scalacheck-shapeless which generates the Gen[A] for you, with completely random values (where you have no control over them).
You may also want to look into random-data-generator (thanks #Gabriele Petronella), which simplifies things even further. From the docs:
import com.danielasfregola.randomdatagenerator.RandomDataGenerator
object MyApp extends RandomDataGenerator {
case class Example(text: String, n: Int)
val example: Example = random[Example]
// Example(ਈ䈦㈾钜㔪旅ꪔ墛炝푰⡨䌆ᵅ퍧咪, 73967257)
}
This is also especially helpful in property based testing.
We've just moved away from scalacheck-shapeless and use Scala/Java reflection instead.
The main reasons are (1) scalacheck-shapeless uses Macros (slow compilation), (2) the API is a bit more verbose than my liking, and (3) the generated values are way too wild (e.g. generating strings with Japanese characters).
However, setting it up is a bit more involved. Here is a full working code that you can copy into your codebase:
import scala.reflect.api
import scala.reflect.api.{TypeCreator, Universe}
import scala.reflect.runtime.universe._
object Maker {
val mirror = runtimeMirror(getClass.getClassLoader)
var makerRunNumber = 1
def apply[T: TypeTag]: T = {
val method = typeOf[T].companion.decl(TermName("apply")).asMethod
val params = method.paramLists.head
val args = params.map { param =>
makerRunNumber += 1
param.info match {
case t if t <:< typeOf[Enumeration#Value] => chooseEnumValue(convert(t).asInstanceOf[TypeTag[_ <: Enumeration]])
case t if t =:= typeOf[Int] => makerRunNumber
case t if t =:= typeOf[Long] => makerRunNumber
case t if t =:= typeOf[Date] => new Date(Time.now.inMillis)
case t if t <:< typeOf[Option[_]] => None
case t if t =:= typeOf[String] && param.name.decodedName.toString.toLowerCase.contains("email") => s"random-$arbitrary#give.asia"
case t if t =:= typeOf[String] => s"arbitrary-$makerRunNumber"
case t if t =:= typeOf[Boolean] => false
case t if t <:< typeOf[Seq[_]] => List.empty
case t if t <:< typeOf[Map[_, _]] => Map.empty
// Add more special cases here.
case t if isCaseClass(t) => apply(convert(t))
case t => throw new Exception(s"Maker doesn't support generating $t")
}
}
val obj = mirror.reflectModule(typeOf[T].typeSymbol.companion.asModule).instance
mirror.reflect(obj).reflectMethod(method)(args:_*).asInstanceOf[T]
}
def chooseEnumValue[E <: Enumeration: TypeTag]: E#Value = {
val parentType = typeOf[E].asInstanceOf[TypeRef].pre
val valuesMethod = parentType.baseType(typeOf[Enumeration].typeSymbol).decl(TermName("values")).asMethod
val obj = mirror.reflectModule(parentType.termSymbol.asModule).instance
mirror.reflect(obj).reflectMethod(valuesMethod)().asInstanceOf[E#ValueSet].head
}
def convert(tpe: Type): TypeTag[_] = {
TypeTag.apply(
runtimeMirror(getClass.getClassLoader),
new TypeCreator {
override def apply[U <: Universe with Singleton](m: api.Mirror[U]) = {
tpe.asInstanceOf[U # Type]
}
}
)
}
def isCaseClass(t: Type) = {
t.companion.decls.exists(_.name.decodedName.toString == "apply") &&
t.decls.exists(_.name.decodedName.toString == "copy")
}
}
And, when you want to use it, you can call:
val user = Maker[User]
val user2 = Maker[User].copy(email = "someemail#email.com")
The code above generates arbitrary and unique values. They aren't exactly randomised. It's best for using in tests.
Read our full blog post here: https://give.engineering/2018/08/24/instantiate-case-class-with-arbitrary-value.html
We've started using Magnolia, which provides a faster type class derivation compared to shapeless for derivation of Arbitrary instances.
Here is the library to use, and here is an example (docs):
case class Inner(int: Int, str: String)
case class Outer(inner: Inner)
// ScalaCheck Arbitrary
import magnolify.scalacheck.auto._
import org.scalacheck._ // implicit instances for Arbitrary[Int], etc.
val arb: Arbitrary[Outer] = implicitly[Arbitrary[Outer]]
arb.arbitrary.sample
// = Some(Outer(Inter(12345, abcde)))

Scalameta: Identify particular annotations

I want to auto-generate REST API models in Scala using scalameta annotation macros. Specifically, given:
#Resource case class User(
#get id : Int,
#get #post #patch name : String,
#get #post email : String,
registeredOn : Long
)
I want to generate:
object User {
case class Get(id: Int, name: String, email: String)
case class Post(name: String, email: String)
case class Patch(name: Option[String])
}
trait UserRepo {
def getAll: Seq[User.Get]
def get(id: Int): User.Get
def create(request: User.Post): User.Get
def replace(id: Int, request: User.Put): User.Get
def update(id: Int, request: User.Patch): User.Get
def delete(id: Int): User.Get
}
I have something working here: https://github.com/pathikrit/metarest
Specifically I am doing this:
import scala.collection.immutable.Seq
import scala.collection.mutable
import scala.annotation.{StaticAnnotation, compileTimeOnly}
import scala.meta._
class get extends StaticAnnotation
class put extends StaticAnnotation
class post extends StaticAnnotation
class patch extends StaticAnnotation
#compileTimeOnly("#metarest.Resource not expanded")
class Resource extends StaticAnnotation {
inline def apply(defn: Any): Any = meta {
val (cls: Defn.Class, companion: Defn.Object) = defn match {
case Term.Block(Seq(cls: Defn.Class, companion: Defn.Object)) => (cls, companion)
case cls: Defn.Class => (cls, q"object ${Term.Name(cls.name.value)} {}")
case _ => abort("#metarest.Resource must annotate a class")
}
val paramsWithAnnotation = for {
Term.Param(mods, name, decltype, default) <- cls.ctor.paramss.flatten
seenMods = mutable.Set.empty[String]
modifier <- mods if seenMods.add(modifier.toString)
(tpe, defArg) <- modifier match {
case mod"#get" | mod"#put" | mod"#post" => Some(decltype -> default)
case mod"#patch" =>
val optDeclType = decltype.collect({case tpe: Type => targ"Option[$tpe]"})
val defaultArg = default match {
case Some(term) => q"Some($term)"
case None => q"None"
}
Some(optDeclType -> Some(defaultArg))
case _ => None
}
} yield modifier -> Term.Param(Nil, name, tpe, defArg)
val models = paramsWithAnnotation
.groupBy(_._1.toString)
.map({case (verb, pairs) =>
val className = Type.Name(verb.stripPrefix("#").capitalize)
val classParams = pairs.map(_._2)
q"case class $className[..${cls.tparams}] (..$classParams)"
})
val newCompanion = companion.copy(
templ = companion.templ.copy(stats = Some(
companion.templ.stats.getOrElse(Nil) ++ models
))
)
Term.Block(Seq(cls, newCompanion))
}
}
I am unhappy with the following snip of code:
modifier match {
case mod"#get" | mod"#put" | mod"#post" => ...
case mod"#patch" => ...
case _ => None
}
The above code does "stringly" pattern matching on the annotations I have. Is there anyway to re-use the exact annotations I have to pattern match for these:
class get extends StaticAnnotation
class put extends StaticAnnotation
class post extends StaticAnnotation
class patch extends StaticAnnotation
It's possible to replace the mod#get stringly typed annotation with a get() extractor using a bit of runtime reflection (at compile time).
In addition, let's say we also want to allow users to fully qualify the annotation with #metarest.get or #_root_.metarest.get
All the following code examples assume import scala.meta._. The tree structure of #get, #metarest.get and #_root_.metarest.get are
# mod"#get".structure
res4: String = """ Mod.Annot(Ctor.Ref.Name("get"))
"""
# mod"#metarest.get".structure
res5: String = """
Mod.Annot(Ctor.Ref.Select(Term.Name("metarest"), Ctor.Ref.Name("get")))
"""
# mod"#_root_.metarest.get".structure
res6: String = """
Mod.Annot(Ctor.Ref.Select(Term.Select(Term.Name("_root_"), Term.Name("metarest")), Ctor.Ref.Name("get")))
"""
The selectors are either Ctor.Ref.Select or Term.Select and the names are either Term.Name or Ctor.Ref.Name.
Let's first create a custom selector extractor
object Select {
def unapply(tree: Tree): Option[(Term, Name)] = tree match {
case Term.Select(a, b) => Some(a -> b)
case Ctor.Ref.Select(a, b) => Some(a -> b)
case _ => None
}
}
Then create a few helper utilities
object ParamAnnotation {
/* isSuffix(c, a.b.c) // true
* isSuffix(b.c, a.b.c) // true
* isSuffix(a.b.c, a.b.c) // true
* isSuffix(_root_.a.b.c, a.b.c) // true
* isSuffix(d.c, a.b.c) // false
*/
def isSuffix(maybeSuffix: Term, fullName: Term): Boolean =
(maybeSuffix, fullName) match {
case (a: Name, b: Name) => a.value == b.value
case (Select(q"_root_", a), b: Name) => a.value == b.value
case (a: Name, Select(_, b)) => a.value == b.value
case (Select(aRest, a), Select(bRest, b)) =>
a.value == b.value && isSuffix(aRest, bRest)
case _ => false
}
// Returns true if `mod` matches the tree structure of `#T`
def modMatchesType[T: ClassTag](mod: Mod): Boolean = mod match {
case Mod.Annot(term: Term.Ref) =>
isSuffix(term, termRefForType[T])
case _ => false
}
// Parses `T.getClass.getName` into a Term.Ref
// Uses runtime reflection, but this happens only at compile time.
def termRefForType[T](implicit ev: ClassTag[T]): Term.Ref =
ev.runtimeClass.getName.parse[Term].get.asInstanceOf[Term.Ref]
}
With this setup, we can add a companion object to the get definition with an
unapply boolean extractor
class get extends StaticAnnotation
object get {
def unapply(mod: Mod): Boolean = ParamAnnotation.modMatchesType[get](mod)
}
Doing the same for post and put, we can now write
// before
case mod"#get" | mod"#put" | mod"#post" => Some(decltype -> default)
// after
case get() | put() | post() => Some(decltype -> default)
Note that this approach will still not work if the user renames for example get on import
import metarest.{get => GET}
I would recommend aborting if an annotation does not match what you expected
// before
case _ => None
// after
case unexpected => abort("Unexpected modifier $unexpected. Expected one of: put, get post")
PS. The object get { def unapply(mod: Mod): Boolean = ... } part is boilerplate that could be generated by some #ParamAnnotation macro annotation, for example #ParamAnnotion class get extends StaticAnnotation

Scala - not a case class nor does it have method .unapply

I am quite new to Scala and got a few unresolved problems with the following code:
object exprs{
println("Welcome to the Scala worksheet")
def show(e: Expr): String = e match {
case Number(x) => x.toString
case Sum(l, r) => show(l) + " + " + show(r)
}
show(Sum(Number(1), Number(44)))
}
trait Expr {
def isNumber: Boolean
def isSum: Boolean
def numValue: Int
def leftOp: Expr
def rightOp: Expr
def eval: Int = this match {
case Number(n) => n
case Sum(e1, e2) => e1.eval + e2.eval
}
}
class Number(n: Int) extends Expr {
override def isNumber: Boolean = true
override def isSum: Boolean = false
override def numValue: Int = n
override def leftOp: Expr = throw new Error("Number.leftOp")
override def rightOp: Expr = throw new Error("Number.rightOp")
}
class Sum(e1: Expr, e2: Expr) extends Expr {
override def isNumber: Boolean = false
override def isSum: Boolean = true
override def numValue: Int = e1.eval + e2.eval
override def leftOp: Expr = e1
override def rightOp: Expr = e2
}
I get the following errors:
Error: object Number is not a case class, nor does it have an unapply/unapplySeq member
Error: not found: value Sum
How to resolve them? Thanks in advance
In Scala case class are like class with extra goodies + some other properties.
For a normal class,
class A(i: Int, s: String)
You can not create its instance like this,
val a = A(5, "five") // this will not work
You will have to use new to create new instance.
val a = new A(5, "five")
Now lets say we have case class,
case class B(i: Int, s: String)
We can create a new instance of B like this,
val b = B(5, "five")
The reason this works with case class is because case class have an auto-created companion objects with them, which provides several utilities including an apply and unapply method.
So, this usage val b = B(5, "five") is actually val b = B.apply(5, "five"). And here B is not the class B but the companion object B which is actually provieds apply method.
Similarly Scala pattern matching uses the unapply (unapplySeq for SeqLike patterns) methods provided by companion object. And hence normal class instances do not work with pattern matching.
Lets say you wanted to defined a class and not a case class for some specific reason but still want to use them with pattern-matching etc, you can provide its companion object with the required methods by yourselves.
class C(val i: Int, val s: String) {
}
object C {
def apply(i: Int, s: String) = new C(i, s)
def unapply(c: C) = Some((c.i, c.s))
}
// now you can use any of the following to create instances,
val c1 = new C(5, "five")
val c2 = C.apply(5, "five")
val c3 = C(5, "five")
// you can also use pattern matching,
c1 match {
case C(i, s) => println(s"C with i = $i and s = $s")
}
c2 match {
case C(i, s) => println(s"C with i = $i and s = $s")
}
Also, as you are new to learning Scala you should read http://danielwestheide.com/scala/neophytes.html which is probably the best resource for any Scala beginner.

Conditional Behavior With Free Monads

I'm following the tutorial here: http://typelevel.org/cats/datatypes/freemonad.html and trying to modify it to work with a cache in front of the key value store. This is what I've come up with so far but I'm getting a compiler error with valueGetOperation. I understand why I get the compile error, I just don't understand how to work around it. What's the best practice for conditional behavior when using a free monad?
import cats.data.Coproduct
import cats.free.{Free, Inject}
object KvStore {
sealed trait KvOp[A]
case class Get[T](key: String) extends KvOp[Option[T]]
case class Put[T](key: String, value: T) extends KvOp[Unit]
case class Delete[T](key: String) extends KvOp[Unit]
}
object CacheStore {
sealed trait CacheOp[A]
case class Get[T](key: String) extends CacheOp[Option[T]]
case class Put[T](key: String, value: T) extends CacheOp[Unit]
case class Delete[T](key: String) extends CacheOp[Unit]
}
type WriteThruCache[A] = Coproduct[KvStore.KvOp, CacheStore.CacheOp, A]
class KvOps[F[_]](implicit I: Inject[KvStore.KvOp, F]) {
import KvStore._
def get[T](key: String): Free[F, Option[T]] = Free.inject[KvOp, F](Get(key))
def put[T](key: String, value: T): Free[F, Unit] = Free.inject[KvOp, F](Put(key, value))
def delete[T](key: String): Free[F, Unit] = Free.inject[KvOp, F](Delete(key))
}
object KvOps {
implicit def kvOps[F[_]](implicit I: Inject[KvStore.KvOp, F]): KvOps[F] = new KvOps[F]
}
class CacheOps[F[_]](implicit I: Inject[CacheStore.CacheOp, F]) {
import CacheStore._
def get[T](key: String): Free[F, Option[T]] = Free.inject[CacheOp, F](Get(key))
def put[T](key: String, value: T): Free[F, Unit] = Free.inject[CacheOp, F](Put(key, value))
def delete[T](key: String): Free[F, Unit] = Free.inject[CacheOp, F](Delete(key))
}
object CacheOps {
implicit def cacheOps[F[_]](implicit I: Inject[CacheStore.CacheOp, F]): CacheOps[F] = new CacheOps[F]
}
def valueWriteOperation[T](implicit Kv: KvOps[WriteThruCache], Cache: CacheOps[WriteThruCache]): ((String, T) => Free[WriteThruCache, Unit]) = {
(key: String, value: T) =>
for {
_ <- Kv.put(key, value)
_ <- Cache.put(key, value)
} yield ()
}
// This is where I'm stuck
// desired behavior: If the value isn't in the cache, load it from the kv store and put it in the cache
def valueGetOperation[T](implicit Kv: KvOps[WriteThruCache], Cache: CacheOps[WriteThruCache]): ((String) => Free[WriteThruCache, Option[T]]) = {
(key: String) =>
for {
cacheOption <- Cache.get[T](key)
kvOption <- Kv.get[T](key) if cacheOption.isEmpty // value withFilter is not a member of cats.free.Free[A$A39.this.WriteThruCache,Option[T]]
} yield cacheOption.orElse(kvOption)
}
As you know in for comprehension, when you use if it is desugared by compiler to calling withFilter method, and if it's not accessible it falls back to filter method. If they are not implemented you will receive compiler error.
However you can simply use if else!
for {
booleanValue <- myfreeAlbebra.checkCondidtion(arg1, arg2)
valueToReturn <- if (booleanValue) {
myfreeAlbebra.someValue
} else {
myfreeAlbebra.someOtherValue
}
} yield valueToReturn
alternatively you can do something like:
for {
booleanValue <- myfreeAlbebra.checkCondidtion(arg1, arg2)
valueToReturnOpt <- myfreeAlbebra.someValue
fallbackValue <- myfreeAlbebra.someOtherValue
} yield valueToReturnOpt.getOrElse(fallbackValue)
The formar one will assign value to valueToReturn depending on booleanValue. As such only one branch will be interpreted. The later will evaluate both values and return one of them depending on whether or not valueToReturnOpt will be empty.
Personally I would try something like:
def valueGetOperation[T](implicit Kv: KvOps[WriteThruCache], Cache: CacheOps[WriteThruCache]): ((String) => Free[WriteThruCache, Option[T]]) = {
(key: String) =>
for {
cacheOption <- Cache.get[T](key)
returnedValue <- if (cacheOption.isEmpty) Cache.get[T](key) else Kv.get[T](key)
} yield returnedValue
}
Following Mateusz' suggestions, this is what I came up with:
def withFallback[A[_], T](loadedValue: Option[T], fallback: => Free[A, Option[T]]): Free[A, Option[T]] = {
if(loadedValue.isDefined) {
Free.pure[A, Option[T]](loadedValue)
} else {
fallback
}
}
def valueGetOperation[T](implicit Kv: KvOps[WriteThruCache], Cache: CacheOps[WriteThruCache]): ((String) => Free[WriteThruCache, Option[T]]) = {
(key: String) =>
for {
cachedOption <- Cache.get[T](key)
actualValue <- withFallback[WriteThruCache, T](cachedOption, fallback = Kv.get[T](key))
} yield actualValue
}
If there's a standard construct to implement withFallback I'd be glad to know about it.
You could also use OptionT#orElse.
import cats.data.OptionT
type KV[A] = Free[WriteThruCache, A]
def valueGetOperation[T](
implicit
Kv: KvOps[WriteThruCache],
Cache: CacheOps[WriteThruCache]
): String => KV[Option[T]] =
key => OptionT[KV, T](Cache.get[T](key)).orElse(OptionT[KV, T](Kv.get[T](key))).value
Or OptionT#orElseF :
def valueGetOperation[T](
implicit
Kv: KvOps[WriteThruCache],
Cache: CacheOps[WriteThruCache]
): String => KV[Option[T]] =
key => OptionT[KV, T](Cache.get[T](key)).orElseF(Kv.get[T](key)).value
Note that with the -Ypartial-unification flag in Scala 2.12 you don't need the KV type alias and you can write OptionT(...) instead of OptionT[KV, T](...).