OUTLINE
I have an API that looks something like this:
package com.example
object ExternalApi {
def create[T <: SpecialElement](elem: T): TypeConstructor[T] =
TypeConstructor(elem)
def create1[T <: SpecialElement](elem: T): TypeConstructor[T] =
TypeConstructor(elem)
def create2[T <: SpecialElement](elem: T): TypeConstructor[T] =
TypeConstructor(elem)
//...
}
object MyApi {
def process[T <: TypeConstructor[_ <: SpecialElement]](
l: T,
metadata: List[String]): T = {
println("I've been called!")
//do some interesting stuff with the List's type parameter here
l
}
}
case class TypeConstructor[E](elem: E)
trait SpecialElement
The ExternalApi (which is actually external to my lib, so no modifying that) has a series of calls that I'd like to automatically wrap with MyApi.process calls, with the metadata argument derived from the final type of T.
To illustrate, the calls to be wrapped could have any form, including nested calls, and calls within other AST subtree types (such as Blocks), e.g. :
package com.example.test
import com.example.{ExternalApi, SpecialElement}
object ApiPluginTest extends App {
//one possible form
val targetList = ExternalApi.create(Blah("name"))
//and another
ExternalApi.create2(ExternalApi.create1(Blah("sth")).elem)
//and yet another
val t = {
val sub1 = ExternalApi.create(Blah("anything"))
val sub2 = ExternalApi.create1(sub1.elem)
sub2
}
}
case class Blah(name: String) extends SpecialElement
Since compiler plugins handle matching structures within ASTs recursively "for free", I've decided to go with them.
However, due to the fact that I need to match a specific type signature, the plugin follows the typer phase.
Here's the code of the PluginComponent:
package com.example.plugin
import com.example.{SpecialElement, TypeConstructor}
import scala.tools.nsc.Global
import scala.tools.nsc.plugins.PluginComponent
import scala.tools.nsc.transform.Transform
class WrapInApiCallComponent(val global: Global)
extends PluginComponent
with Transform {
protected def newTransformer(unit: global.CompilationUnit) =
WrapInApiCallTransformer
val runsAfter: List[String] = List("typer") //since we need the type
val phaseName: String = WrapInApiCallComponent.Name
import global._
object WrapInApiCallTransformer extends Transformer {
override def transform(tree: global.Tree) = {
val transformed = super.transform(tree)
transformed match {
case call # Apply(_, _) =>
if (call.tpe != null && call.tpe.finalResultType <:< typeOf[
TypeConstructor[_ <: SpecialElement]]) {
println(s"Found relevant call $call")
val typeArguments = call.tpe.typeArgs.map(_.toString).toList
val listSymbOf = symbolOf[List.type]
val wrappedFuncSecondArgument =
q"$listSymbOf.apply(..$typeArguments)"
val apiObjSymbol = symbolOf[com.example.MyApi.type]
val wrappedCall =
q"$apiObjSymbol.process[${call.tpe.finalResultType}]($call, $wrappedFuncSecondArgument)"
//explicit typing, otherwise later phases throw NPEs etc.
val ret = typer.typed(wrappedCall)
println(showRaw(ret))
println("----")
ret
} else {
call
}
case _ => transformed
}
}
}
}
object WrapInApiCallComponent {
val Name = "api_embed_component"
}
This seems to resolve identifiers, as well as types, correctly, outputting e.g.:
Apply(TypeApply(Select(TypeTree().setOriginal(Ident(com.example.MyApi)), TermName("process")), List(TypeTree())), List(Apply(TypeApply(Select(Select(Select(Ident(com), com.example), com.example.MyApi), TermName("create")), List(TypeTree())), List(Apply(Select(Ident(com.example.test.Blah), TermName("apply")), List(Literal(Constant("name")))))), Apply(TypeApply(Select(TypeTree().setOriginal(Ident(scala.collection.immutable.List)), TermName("apply")), List(TypeTree())), List(Literal(Constant("com.example.test.Blah"))))))
Unfortunately, I get an error during compilation starting with:
scala.reflect.internal.FatalError:
[error]
[error] Unexpected tree in genLoad: com.example.MyApi.type/class scala.reflect.internal.Trees$TypeTree at: RangePosition([projectpath]/testPluginAutoWrap/compiler_plugin_test/src/main/scala/com/example/test/ApiPluginTest.scala, 108, 112, 112)
[error] while compiling: [projectpath]/testPluginAutoWrap/compiler_plugin_test/src/main/scala/com/example/test/ApiPluginTest.scala
[error] during phase: jvm
[error] library version: version 2.12.4
[error] compiler version: version 2.12.4
[error] reconstructed args: -Xlog-implicits -classpath [classpath here]
[error]
[error] last tree to typer: TypeTree(class String)
[error] tree position: line 23 of [projectpath]/testPluginAutoWrap/compiler_plugin_test/src/main/scala/com/example/test/ApiPluginTest.scala
QUESTION
It looks I'm screwing something up with type definitions, but what is it?
Specifically:
How do I correctly wrap every ExternalApi.createX call with an MyApi.process call, constrained by the requirements provided above?
NOTES
Given the amount of boilerplate required, I've set up a complete example project. It's available here.
The answer does not have to define a compiler plugin. If you're able to cover all the relevant calls with a macro, that is fine as well.
Originally the wrapped call was to something like: def process[T <: TypeConstructor[_ <: SpecialElement] : TypeTag](l: T): T, the setup here is actually a workaround. So if you are able to generate a wrapped call of this type, i.e. one that includes a runtime TypeTag[T], that's fine as well.
Related
Scala application use case:
We have a Scala based that module reads the data from global cache (Redis) and save the same into local cache(Caffeine LoadingCache). As we want this data to be refreshed asynchronously, we are using LoadingCache with refreshAfterWrite duration set to refresh window of 2.second.
Question:
Not question but need help with the following code that is giving warning and also compile time errors
Warning: For build method, it gives warning as Implements member load in CacheLoader (com.github.benmanes.caffeine.cache)
Compile time error 1: type arguments [Int,redisToCaffeine.DataObject] conform to the bounds of none of the overloaded alternatives of value build: [K1 <: Object, V1 <: Object](x$1: com.github.benmanes.caffeine.cache.CacheLoader[_ >: K1, V1])com.github.benmanes.caffeine.cache.LoadingCache[K1,V1] <and> [K1 <: Object, V1 <: Object]()com.github.benmanes.caffeine.cache.Cache[K1,V1] .build[Int, DataObject](key => loader(key))
Compile time error 2: wrong number of type parameters for overloaded method value build with alternatives: [K1 <: Object, V1 <: Object](x$1: com.github.benmanes.caffeine.cache.CacheLoader[_ >: K1, V1])com.github.benmanes.caffeine.cache.LoadingCache[K1,V1] <and> [K1 <: Object, V1 <: Object]()com.github.benmanes.caffeine.cache.Cache[K1,V1] .build[Int, DataObject](key => loader(key))
Code:
package redisToCaffeine
import scala.concurrent.duration._
import com.github.benmanes.caffeine.cache.{ CacheLoader, Caffeine, LoadingCache }
import com.twitter.finagle.stats.InMemoryStatsReceiver
import javax.annotation.Nullable
import redisToCaffeine.CacheImplicits.StatsReportingCaffeineCache
class LocalDealService {
class DataObject(data: String) {
override def toString: String = {
"[ 'data': '" + this.data + "' ]"
}
}
val defaultCacheExpireDuration: FiniteDuration = 2.second
val stats: InMemoryStatsReceiver = new InMemoryStatsReceiver
// loader helper
#Nullable
#throws[Exception]
protected def loader(key: Int): DataObject = { // this will replace to read the data from Redis Cache
new DataObject(s"LOADER_HELPER_$key")
}
def initCache(maximumSize: Int = 5): LoadingCache[Int, DataObject] = {
Caffeine
.newBuilder()
.maximumSize(maximumSize)
.refreshAfterWrite(defaultCacheExpireDuration.length, defaultCacheExpireDuration.unit)
.recordStats()
.build[Int, DataObject](key => loader(key))
.enableCacheStatsReporting("deal-service", stats)
}
}
I'm new to Scala and Caffeine both so not sure what I'm be doing wrong; I tried different ways mentioned here and here to write loader but nothing worked (mainly they are in Java). Little research around Scala bounds doesn't helped here any way. Kindly help.
Depends on which Scala version is being used here.
Although Scala (2.12 and later) Functions support conversions to Java SAM, these are done only when explicitly required. So if you are using Scala 2.12 or later, you can explicitly ask the compiler to convert the Scala function to SAM,
Also, don't use Int as key for the cache. Although it will work because of implicit conversions to Integer, that is not a good practice.
def initCache(maximumSize: Int = 5): LoadingCache[Integer, DataObject] = {
Caffeine
.newBuilder()
.maximumSize(maximumSize)
.refreshAfterWrite(defaultCacheExpireDuration.length, defaultCacheExpireDuration.unit)
.recordStats()
.build[Integer, DataObject]((key => loader(key)): CacheLoader[Integer, DataObject])
.enableCacheStatsReporting("deal-service", stats)
}
And if you are dealing with older Scala versions, then just forget that SAM exists and do it old style.
def initCache(maximumSize: Int = 5): LoadingCache[Integer, DataObject] = {
Caffeine
.newBuilder()
.maximumSize(maximumSize)
.refreshAfterWrite(defaultCacheExpireDuration.length, defaultCacheExpireDuration.unit)
.recordStats()
.build[Int, DataObject](new CacheLoader[Integer, DataObject] {
override def load(key: Integer): DataObject = loader(key)
})
.enableCacheStatsReporting("deal-service", stats)
}
I'm using sbt to build some of the riscv boom from the source code, but the sbt complains that it "could not find implicit value for parameter valName: : freechips.rocketchip.diplomacy.ValName". The detailed error message are as below:
[error] F:\hiMCU\my_proj\src\main\scala\freechips\rocketchip\tile\BaseTile.scala:170:42: could not find implicit value for parameter valName: freechips.rocketchip.diplomacy.ValName
[error] Error occurred in an application involving default arguments.
[error] protected val tlMasterXbar = LazyModule(new TLXbar)
The code where sbt complains is as below:
abstract class BaseTile private (val crossing: ClockCrossingType, q: Parameters)
extends LazyModule()(q)
with CrossesToOnlyOneClockDomain
with HasNonDiplomaticTileParameters
{
// Public constructor alters Parameters to supply some legacy compatibility keys
def this(tileParams: TileParams, crossing: ClockCrossingType, lookup: LookupByHartIdImpl, p: Parameters) = {
this(crossing, p.alterMap(Map(
TileKey -> tileParams,
TileVisibilityNodeKey -> TLEphemeralNode()(ValName("tile_master")),
LookupByHartId -> lookup
)))
}
def module: BaseTileModuleImp[BaseTile]
def masterNode: TLOutwardNode
def slaveNode: TLInwardNode
def intInwardNode: IntInwardNode // Interrupts to the core from external devices
def intOutwardNode: IntOutwardNode // Interrupts from tile-internal devices (e.g. BEU)
def haltNode: IntOutwardNode // Unrecoverable error has occurred; suggest reset
def ceaseNode: IntOutwardNode // Tile has ceased to retire instructions
def wfiNode: IntOutwardNode // Tile is waiting for an interrupt
protected val tlOtherMastersNode = TLIdentityNode()
protected val tlSlaveXbar = LazyModule(new TLXbar)
protected val tlMasterXbar = LazyModule(new TLXbar)
protected val intXbar = LazyModule(new IntXbar)
....
}
The LazyModule object code is as below:
object LazyModule
{
protected[diplomacy] var scope: Option[LazyModule] = None
private var index = 0
def apply[T <: LazyModule](bc: T)(implicit valName: ValName, sourceInfo: SourceInfo): T = {
// Make sure the user put LazyModule around modules in the correct order
// If this require fails, probably some grandchild was missing a LazyModule
// ... or you applied LazyModule twice
require (scope.isDefined, s"LazyModule() applied to ${bc.name} twice ${sourceLine(sourceInfo)}")
require (scope.get eq bc, s"LazyModule() applied to ${bc.name} before ${scope.get.name} ${sourceLine(sourceInfo)}")
scope = bc.parent
bc.info = sourceInfo
if (!bc.suggestedNameVar.isDefined) bc.suggestName(valName.name)
bc
}
}
I think the sbt should find some val of type freechips.rocketchip.diplomacy.ValName, but it didn't find such kind of val.
You need to have an object of type ValName in the scope where your LazyModules are instantiated:
implicit val valName = ValName("MyXbars")
For more details on Scala's implicit please see https://docs.scala-lang.org/tutorials/tour/implicit-parameters.html.html
You generally shouldn't need to manually create a ValName, the Scala compiler can materialize them automatically based on the name of the val you're assigning the LazyModule to. You didn't include your imports in your example, but can you try importing ValName?
import freechips.rocketchip.diplomacy.ValName
In most of rocket-chip code, this is imported via wildcard importing everything in the diplomacy package
import freechips.rocketchip.diplomacy._
Background
I'm trying to add a #Prisms annotation to Monocle that will work as follows. Given:
#Prisms sealed trait Foo
case object A extends Foo
case object B extends Foo
it will generate a companion object for Foo:
object Foo {
val a = monocle.macros.GenPrism.apply[Foo, A]
val b = monocle.macros.GenPrism.apply[Foo, B]
}
(or if the companion object already exists, it will add those methods to it).
The macro relies on directKnownSubclasses to find A and B, so there will be a note in the Monocle readme recommending that people use Typelevel Scala. I've configured Monocle to use TLS on my branch.
First attempt
I tried using TypeNames to represent the parent type Foo and child type (A or B):
def findSubclasses(typeName: TypeName): Set[TypeName] = {
// Note: disable macros to prevent stack overflow caused by infinite typing loop!
val tpe = c.typecheck(Ident(typeName), mode = c.TYPEmode, silent = true, withMacrosDisabled = true)
tpe.symbol.asClass.knownDirectSubclasses.map(_.asType.name)
}
def prisms(parentTypename: TypeName, childNames: Set[TypeName]): Set[Tree] = childNames.map { childName =>
val prismName = TermName(prefix + childName.decodedName.toString.toLowerCase)
q"""val $prismName = monocle.macros.GenPrism.apply[$parentTypename, $childName]"""
}
This generates what looks like reasonable code:
{
sealed trait Foo;
object Foo {
val a = monocle.macros.GenPrism.apply[Foo, A];
val b = monocle.macros.GenPrism.apply[Foo, B]
};
()
}
But it doesn't seem to work:
[error] /Users/chris/code/Monocle/test/shared/src/test/scala/other/PrismsAnnotationSpec.scala:5: not found: type A
[error] #monocle.macros.Prisms sealed trait Foo
[error] ^
[error] one error found
Second attempt
I also tried using TypeSymbols instead of TypeNames:
def prisms(parentTypename: TypeName, childSymbols: Set[TypeSymbol]): Set[Tree] = {
val parentSymbol: TypeSymbol =
c.typecheck(Ident(parentTypename), mode = c.TYPEmode, silent = true, withMacrosDisabled = true)
.symbol
.asType
childSymbols.map { childSymbol =>
val prismName = TermName(prefix + childSymbol.name.decodedName.toString.toLowerCase)
q"""val $prismName = monocle.macros.GenPrism.apply[$parentSymbol, $childSymbol]"""
}
}
But this gives me the following error, so I'm guessing the generated tree is invalid:
[error] /Users/chris/code/Monocle/test/shared/src/test/scala/other/PrismsAnnotationSpec.scala:5: macro annotation could not be expanded (the most common reason for that is that you need to enable the macro paradise plugin; another possibility is that you try to use macro annotation in the same compilation run that defines it)
[error] #monocle.macros.Prisms sealed trait Foo
[error] ^
[error] one error found
Third attempt
I also tried turning the TypeSymbols into Types:
def prisms(parentTypename: TypeName, childSymbols: Set[TypeSymbol]): Set[Tree] = {
val parentSymbol: TypeSymbol =
c.typecheck(Ident(parentTypename), mode = c.TYPEmode, silent = true, withMacrosDisabled = true)
.symbol
.asType
childSymbols.map { childSymbol =>
val prismName = TermName(prefix + childSymbol.name.decodedName.toString.toLowerCase)
q"""val $prismName = monocle.macros.GenPrism.apply[${parentSymbol.toType}, ${childSymbol.toType}]"""
}
}
But I get the same error, saying the annotation could not be expanded.
Now I'm out of ideas.
How to reproduce
The complete code is available on this branch: https://github.com/cb372/Monocle/tree/sealed-trait-macro-annotation.
First attempt
Second attempt
Third attempt
The macro is used in test/shared/src/test/scala/other/PrismsAnnotationSpec.scala. To compile it, run sbt test:compile.
I'm trying to implement basically the same thing that is discussed here, but in my specific situation, it's not working.
Currently I have a function that validates a JSON response from our server. The problem is, it has the JSON type hardcoded into the method:
def fakeRequest[A: Writes](target: () => Call, requestObject: A): Any = {
route(FakeRequest(target()).withJsonBody(Json.toJson(requestObject))) match {
// ... stuff happens
Json.parse(contentAsString(response)).validate[GPInviteResponse]
^
Note the hardcoded GPInviteResponse type.
So, to make this a totally generic and reusable method, it would be great to pass in the type that is being validated.
I tried this:
def fakeRequest[A: Writes, B](target: () => Call, requestObject: A, responseType: B): Any = {
route(FakeRequest(target()).withJsonBody(Json.toJson(requestObject))) match {
// ... stuff happens
Json.parse(contentAsString(response)).validate[B]
^
That almost works, but I get a No Json deserializer found for type B. Makes sense, so I changed it to:
def fakeRequest[A: Writes, B: Reads](target: () => Call, requestObject: A, responseType: B): Any = {
But, now I get No Json deserializer found for type controllers.GPInviteResponse.type. So the question is: Is it possible to pass a type like this (or is there some other magic to make it work)?
There is a deserializer defined for the type... I've re-read it half a dozen times to make sure there's no typo:
case class GPInviteResponse(inviteSent: Boolean, URL: Option[String], error: Option[GPRequestError] = None) {
def this(error: GPRequestError) = this(false, None, Option(error))
}
object GPInviteResponse {
implicit val readsInviteResponse = Json.reads[GPInviteResponse]
implicit val writesInviteResponse = Json.writes[GPInviteResponse]
}
Edit
Included below is a simplified test case that demonstrates the problem. At the moment, this won't compile (error is shown below). I think I understand why it won't work (vaguely) but I have no idea regarding a solution. My theory as to why it won't work: While the provided type GPInviteRequest does have an implicit Reads/Writes method, Scala cannot make the connection that an instance B is actually a GPInviteRequest so it concludes that B does not have Reads/Writes.
case class GPInviteResponse(inviteSent: Boolean)
object GPInviteResponse {
implicit val readsInviteResponse = Json.reads[GPInviteResponse]
implicit val writesInviteResponse = Json.writes[GPInviteResponse]
}
class TestInviteServices extends PlaySpecification {
"try to validate a type" in {
tryToValidate(GPInviteRequest(true))
}
def tryToValidate[B: Reads, Writes](i: B) = {
val json = Json.toJson(i).toString
Json.parse(json).validate[B].isSuccess must beTrue
}
}
The above test yields:
[error]
/Users/zbeckman/Projects/Glimpulse/Server-2/project/glimpulse-server/test/application/TestInviteServices.scala:46:
No Json serializer found for type B. Try to implement an implicit
Writes or Format for this type. [error] val json =
Json.toJson(i).toString [error] ^ [error]
/Users/zbeckman/Projects/Glimpulse/Server-2/project/glimpulse-server/test/application/TestInviteServices.scala:133:
No Json deserializer found for type controllers.GPInviteResponse.type.
Try to implement an implicit Reads or Format for this type. [error]
fakeRequest(controllers.routes.GPInviteService.invite, i,
GPInviteResponse) match { [error] ^
Your error message:
No Json serializer found for type B. Try to implement an implicit Writes or Format for this type.
In this function, how is the toJson method supposed to know how to serialize your type B?
def tryToValidate[B: Reads, Writes](i: B) = {
val json = Json.toJson(i).toString
Json.parse(json).validate[B].isSuccess must beTrue
}
You haven't pulled in a writer/reader into scope, the compiler doesn't know where to look for one and that's why it's telling you to implement one. Here's a quick solution
case class GPInviteResponse(inviteSent: Boolean)
object GPInviteResponse {
implicit val format = Json.format[GPInviteResponse]
}
def tryToValidate[B](i: B)(implicit format: Format[B]) = {
val json = Json.toJson(i).toString
Json.parse(json).validate[B]
}
Note: using the format method is equivalent to defining both a reads and writes.
Now, there is an implicit formatter for B in scope, so the compiler knows where to find it and injects it into the validate method which takes a reader implicitly:
// From play.api.libs.json
def validate[T](implicit rds: Reads[T]): JsResult[T] = rds.reads(this)
Edit
You can add type parameters to your function and then reference them in the validate[T] method, like so:
// Define another case class to use in the example
case class Foo(bar: String)
object Foo {
implicit val format = Json.format[Foo]
}
def tryToValidate[B, C](implicit f1: Format[B], f2: Format[C]) = {
val j1 = """{"inviteSent":true}"""
val j2 = """{"bar":"foobar"}""" //
Json.parse(j1).validate[B]
Json.parse(j2).validate[C]
}
// Example call
tryToValidate[GPInviteResponse, Foo]
Try it this way :
def tryToValidate[B](i: B)(implicit format : Format[B]) = {
val json = Json.toJson(i).toString
Json.parse(json).validate[B].isSuccess must beTrue
}
I have following macro (it finds startup objects - project is split across several sbt modules and some of them might need initialization):
object FinderMacro {
def inject[A <: {def self(): A}]: Seq[A] = macro injectImpl[A]
def injectImpl[A : c.WeakTypeTag](c: Context): c.Expr[Seq[A]] = {
import c.universe._
val tpe = c.weakTypeOf[A].typeSymbol
val classess = c.mirror.staticPackage("<empty>").typeSignature.declarations.collect {
case symbol if symbol.typeSignature.baseClasses.contains(tpe) =>
Apply(Select(TypeTree(symbol.typeSignature), newTermName("self")), Nil)
}.toList
c.Expr[Seq[A]](Apply(Select(reify(Seq).tree, newTermName("apply")), classess))
}
}
But it's not working:
Unexpected tree in genLoad: XXX.type/class scala.reflect.internal.Trees$TypeTree ...
followed by tree of macro invoker.
What should I do to get rid of this error?
EDIT:
I solved this problem by changing boot objects to classes and instancing them with constructor:
Apply(Select(New(Ident(symbol)), nme.CONSTRUCTOR), Nil)