Scala Map to Arbitrary Class (Scala Reflection) - scala

If I have a Scala Map
val map = Map("a" -> true,
"b" -> "hello"
"c" -> 5)
Is there a way I can convert this to an object (case class, regular class, anything really) such that I can access the field like this:
val obj = map.toObj
println(obj.a)
println(obj.b)
Without knowing what the parameters will be ahead of time?
Or even better, can it be an instance variable of the class I'm currently working with?
So I could actually just have
println(a)
println(b)

Have a look at Dynamic:
import scala.language.dynamics
class Wrapper(m: Map[String, Any]) extends Dynamic {
def selectDynamic(name: String) = {
m(name)
}
}
object Demo {
def main(args: Array[String]) {
val map = Map("a" -> true,
"b" -> "hello",
"c" -> 5)
val w = new Wrapper(map)
println(w.a)
println(w.b)
}
}
Dynamic gives you "properties on demand".
But it's not type-safe. In the snippet above, if you try to access a non-existing key in the map, a NoSuchElementException will be thrown. And you get the Any type, so you have to use asInstanceOf at the caller's side, for example.
Dynamic has further methods, so have a closer look at the documentation of Dynamic

I hope you understand by asking this question that you will loose all compile time guarantees when you rely on runtime data. If this is really what you want then you can rely on Dynamic:
object X extends App {
import scala.language.dynamics
class D(fields: Map[String, Any]) extends Dynamic {
def selectDynamic(str: String): Any =
fields.getOrElse(str, throw new NoSuchFieldException(str))
}
val fields = Map[String, Any]("a" -> true, "b" -> "hello")
val obj = new D(fields)
println(obj.a)
println(obj.b)
}
Dynamic is a compiler feature that translates all field/method calls to a call to the *Dynamic* methods. Because the compiler can't know anything about your program (how should it?), the return type you get here is Any and when you call a field/method that does not exist you get an exception at runtime instead of a compile time error. When some fields/methods are known to compile time you can combine Dynamic with macros to get at least some compile time checking (such a method is described in the linked answer).
Beside from that, that is the only syntax you can enable in Scala. If return types are important to you, you can at least add them as type parameter:
object X extends App {
import scala.language.dynamics
class D(fields: Map[String, Any]) extends Dynamic {
def selectDynamic[A : reflect.ClassTag](str: String): A =
fields.get(str) match {
case Some(f: A) => f
case _ => throw new NoSuchFieldException(str)
}
}
val fields = Map[String, Any]("a" -> true, "b" -> "hello")
val obj = new D(fields)
println(obj.a[Boolean])
println(obj.b[String])
}

Related

Is it possible to call a scala macro from generic scala code?

I'm trying to use Scala macros to convert untyped, Map[String, Any]-like expressions to their corresponding typed case class expressions.
The following scala macro (almost) gets the job done:
trait ToTyped[+T] {
def apply(term: Any): T
}
object TypeConversions {
// At compile-time, "type-check" an untyped expression and convert it to
// its appropriate typed value.
def toTyped[T]: ToTyped[T] = macro toTypedImpl[T]
def toTypedImpl[T: c.WeakTypeTag](c: Context): c.Expr[ToTyped[T]] = {
import c.universe._
val tpe = weakTypeOf[T]
if (tpe <:< typeOf[Int] || tpe <:< typeOf[String]) {
c.Expr[ToTyped[T]](
q"""new ToTyped[$tpe] {
def apply(term: Any): $tpe = term.asInstanceOf[$tpe]
}""")
} else {
val companion = tpe.typeSymbol.companion
val maybeConstructor = tpe.decls.collectFirst {
case m: MethodSymbol if m.isPrimaryConstructor => m
}
val constructorFields = maybeConstructor.get.paramLists.head
val subASTs = constructorFields.map { field =>
val fieldName = field.asTerm.name
val fieldDecodedName = fieldName.toString
val fieldType = tpe.decl(fieldName).typeSignature
q"""
val subTerm = term.asInstanceOf[Map[String, Any]]($fieldDecodedName)
TypeConversions.toTyped[$fieldType](subTerm)
"""
}
c.Expr[ToTyped[T]](
q"""new ToTyped[$tpe] {
def apply(term: Any): $tpe = $companion(..$subASTs)
}""")
}
}
}
Using the above toTyped function, I can convert for example an untyped person value to its corresponding typed Person case class:
object TypeConversionTests {
case class Person(name: String, age: Int, address: Address)
case class Address(street: String, num: Int, zip: Int)
val untypedPerson = Map(
"name" -> "Max",
"age" -> 27,
"address" -> Map("street" -> "Palm Street", "num" -> 7, "zip" -> 12345))
val typedPerson = TypeConversions.toTyped[Person](untypedPerson)
typedPerson shouldEqual Person("Max", 27, Address("Palm Street", 7, 12345))
}
However, my problem arises when trying to use the toTyped macro from above in generic scala code. Suppose I have a generic function indirection that uses the toTyped macro:
object CanIUseScalaMacrosAndGenerics {
def indirection[T](value: Any): T = TypeConversions.toTyped[T](value)
import TypeConversionTests._
val indirectlyTyped = indirection[Person](untypedPerson)
indirectlyTyped shouldEqual Person("Max", 27, Address("Palm Street", 7, 12345))
Here, I get a compile-time error from the toTyped macro complaining that the type T is not yet instantiated with a concrete type. I think the reason for the error is that from the perspective of toTyped inside indirection, the type T is still generic and not inferred to be Person just yet. And therefore the macro cannot build the corresponding Person case class when called via indirection. However, from the perspective of the call-site indirection[Person](untypedPerson), we have T == Person, so I wonder if there is a way to obtain the instantiated type of T (i.e., Person) inside the macro toTyped.
Put differently: Can I combine the Scala macro toTyped with the generic function indirection and yet be able to figure out the instantiated type for type parameter T inside the toTyped macro? Or am I on a hopeless track here and there is no way to combine Scala macros and generics like this? In the latter case I would like to know if the only solution here is to push the macro usage so far "out" that I can call it instantiated as toTyped[Person] rather than as toTyped[T].
Any insights are very much appreciated. Thank you! :-)
Macros need to be expanded. Every time you use a function which body is a macro, Scala will have to generate the code and put it there. As you suspect, this is very very specific and contradict the idea of parametric polymorphism where you write code independent of specific knowledge about your type.
Type classes are one of solutions to the general problem when you want to have one generic (parametric) definition and multiple per-type implementations of certain parts of your algorithm. You basically, define something that you could consider interface which (most likely) need to follow some contract (speaking in OOP terminology) and pass this interface as as argument:
// example
trait SpecificPerType[T] {
def doSomethingSpecific(t: T): String
}
val specificForString: SpecificPerType[String] = new SpecificPerType[String] {
def doSomethingSpecific(t: String): String = s"MyString: $t"
}
val specificForInt: SpecificPerType[Int] = new SpecificPerType[Int] {
def doSomethingSpecific(t: Int): String = s"MyInt: $t"
}
def genericAlgorithm[T](values: List[T])(specific: SpecificPerType[T]): String =
values.map(specific.doSomethingSpecific).mkString("\n")
genericAlgorithm(List(1,2,3))(specificForInt)
genericAlgorithm(List("a","b","c"))(specificForString)
As you can see, it would be pretty annoying to pass this specific part around, which is one of the reasons implicits were introduced.
So you could write it using implicits like this:
implicit val specificForString: SpecificPerType[String] = new SpecificPerType[String] {
def doSomethingSpecific(t: String): String = s"MyString: $t"
}
implicit val specificForInt: SpecificPerType[Int] = new SpecificPerType[Int] {
def doSomethingSpecific(t: Int): String = s"MyInt: $t"
}
def genericAlgorithm[T](values: List[T])(implicit specific: SpecificPerType[T]): String =
values.map(specific.doSomethingSpecific).mkString("\n")
/* for implicits with one type parameter there exist a special syntax
allowing to express them as if they were type constraints e.g.:
def genericAlgorithm[T: SpecificPerType](values: List[T]): String =
values.map(implicitly[SpecificPerType[T]].doSomethingSpecific).mkString("\n")
implicitly[SpecificPerType[T]] is a summoning that let you access implicit
by type, rather than by its variable's name
*/
genericAlgorithm(List(1,2,3)) // finds specificForString using its type
genericAlgorithm(List("a","b","c")) // finds specificForInt using its type
If you generate that trait implementation using macro, you will be able to have a generic algorithm e.g.:
implicit def generate[T]: SpecificPerType[T] =
macro SpecificPerTypeMacros.impl // assuming that you defined this macro there
As far as I can tell, this (extracting macros into type classes) is one of common patterns when it comes to being
able to generate some code with macros while, still building logic on top of it
using normal, parametric code.
(Just to be clear: I do not claim that the role of type classes is limited as the carriers of macro generated code).

In scalajs with scalajs-react how do I pass a scalajs defined component to a javascript defined component?

My issue is also here:
https://gist.github.com/somanythings/8c3d34de754af311d7826ea837d160b4
In using scalajs with japgolly's scalajs-react (https://github.com/japgolly/scalajs-react) library. I am trying to wrap the griddle grid http://griddlegriddle.github.io/Griddle/customization.html
I want a custom column, and to do that requires I pass a columnMetadata structure which includes a component.
When I do, I can render a scalajs defined component that doesn't have properties, but if I try to access properties through renderP, or scope through renderS, they are both undefined in the scope of the render function. If I debug in the browser, they names are bound, and do have the expected values.
When I break on
def renderP(f: (DuringCallbackU[P, S, B], P) => ReactElement): Out =
render($ => f($, $.props))
then $.props is undefined
What am I missing? Is it a simple typing issue in the ReactComponentB dispatching.
Is it somehow related to https://github.com/japgolly/scalajs-react/issues/157, and I just haven't seen how?
// Confusion over how to pass a scalajs defined component to a javascript defined component
object GriddleComponentWrapper {
// for customComponent I've tried js.Any, ReactComponentU
#ScalaJSDefined
class ColumnMeta(val columnName: String, val order: Int, val customComponent: ReactClass=null) extends js.Object
}
case class GriddleComponentWrapper(results: js.Any, //Seq[Map[String, Any]],
columns: Seq[String],
columnMeta: Option[Seq[ColumnMeta]] = None,
showSettings: Boolean = true,
showFilter: Boolean = true
) {
def toJS = {
val p = js.Dynamic.literal()
p.updateDynamic("results")(results)
p.updateDynamic("columns")(columns)
p.updateDynamic("showSettings")(showSettings)
p.updateDynamic("showFilter")(showFilter)
(columnMeta).foreach { case cm => p.updateDynamic("columnMetadata")(cm.toJsArray) }
p
}
def apply(children: ReactNode*) = {
val f = React.asInstanceOf[js.Dynamic].createFactory(js.Dynamic.global.Bundle.griddle) // access real js component , make sure you wrap with createFactory (this is needed from 0.13 onwards)
f(toJS, children.toJsArray).asInstanceOf[ReactComponentU_]
}
}
object MyTestGrid {
#js.native
class ColumnMetaProps(val data: js.Object, val rowData: js.Object, val metadata: js.Object) extends js.Object
// I've tried making the Props argument js.Dynamic, and also the ColumnMetaProps above
#JSExport
val testComp = ReactComponentB[js.Dynamic]("Mine").renderP(
(sc, props: js.Dynamic) => {
//when debugging this in the browser, 'sc' and 'props' have inspectable object values with the expected members in the browser
//dev tools, BUT, they're undefined
log.info(s"what is ${sc.props}")
log.info(s"what is $props")
val string: Frag = if (!js.isUndefined(props)) props.data.toString() else "nothing!"
<.h1(string)
}).build
#JSExport
val aCompletelyStaticComponentWithNoPropsWillWork = ReactComponentB[js.Dynamic]("MyStaticComponent").renderP(
(sc, props: js.Dynamic) => <.h1("this renders!!") ).build
// am I passing the right thing to columnmeta with testComp.reactClass?
val columnMeta = (new ColumnMeta("c1", 1, testComp.reactClass) :: Nil).toJsArray
val results = Seq(
js.Dynamic.literal("c1" -> "row1c1", "c2" -> "row1c2"),
).toJsArray
val component = ReactComponentB[js.Dynamic]("MyTestGrid")
.render_P {
props =>
GriddleComponentWrapper(results, columns = "c1" :: "c2" :: Nil, columnMeta = Some(columnMeta))()
}.build
def apply() = component
}
React.JS requires that props and state always be an object (or null). Using a single Scala value like a primitive or case class instance, causes exceptions in React.JS. Therefore in scalajs-react, in order to allow users to use any props/state types in a type-safe manner, under the hood an object with a single key of "v" is used. I.e. instead of using 123 directly, {v:123} is used.
You likely will need to accommodate that boxing in your code.
Now, in the next major version (see the "neo" branch), the representations of components vastly improved such that there is no more hidden magic like I just described. Whereas v0.x wasn't designed to facilitate JSā†”Scala component interop, it will be explicit, obvious, and hopefully trivial in neo.

scala typing require implicit

I'm trying to build following
I have a parent generic class
abstract class ResultProvider[+T: Writes](db: DB) {
def get(id: Long): Future[Seq[T]]
}
And some implementations, e.g.
class LengthProvider(db: DB) extends ResultProvider[LengthResult](db){
override def get (userId: Long): Future[Seq[LengthResult]] = ...
}
object LengthProvider extends ((DB) => DisciplinePredictor) {
override def apply(db: DB) = new LengthProvider(db)
}
I have following configuration map:
val providers: Map[String, ((DB) => ResultProvider[???])] = Map(
"length" -> LengthProvider,
"width" -> WidthProvider,
...
)
My question is what should I put in place of ???. Ideally, it should be something like T : Writes, as I only care that this type has Writes implicit implemented, as I'm going to Json.toJson it. It will compile with Any, but then the information that the class should implement Writes implicit is lost.
Or should I use a different approach? I could probably create a superclass for all my result case classes (e.g.LengthResult), but I want to get away with the implicits.
You should be able to write ResultProvider[_] (search for "existential types" if you are unfamiliar with this syntax), but you'll need to give a name to the implicit:
abstract class ResultProvider[+T](db: DB)(implicit val writes: Writes[T]) { ... }
Elsewhere:
val provider: ResultProvider[_] = providers("length")
import provider.writes // makes the implicit visible here
...
You might need to help the compiler (or yourself, if you need to name the type) by providing a type variable:
providers("length") match {
case provider: ResultProvider[a] =>
import provider.writes
...
}

Macro to map "items" to their type parameterized "containers" -- large compiler error

Suppose I have a trait for "items" and another trait for "containers" of those items:
sealed trait Item
case class Marble() extends Item
case class Book() extends Item
sealed trait Container[Item]
case object Pouch extends Container[Marble]
case object Shelf extends Container[Book]
Using the type information above, I'd like to construct a map like so via a macro and some runtime reflection:
// Map(classOf[Marble] -> Pouch, classOf[Book] -> Shelf)
val itemToContainer = typeParamMap[Container[_], Item](Pouch, Shelf)
Here's my attempt, which compiles fine, and works in an intermediate form that just returns the list of TypeTags:
object Macros {
def getTag[T](implicit ttag: ru.TypeTag[T]) = ttag
def getBaseParam(child: ru.Type, base: ru.Symbol, paramBase: ru.Symbol) =
child.baseType(base)
.find(_.baseClasses.contains(paramBase))
.get
.typeSymbol
.asClass // TODO how to get java.lang.Class from here?
def typeParamMapImpl[B, P](c: Context)(children: c.Expr[B]*)(baseTag: c.Expr[ru.WeakTypeTag[B]], paramTag: c.Expr[ru.WeakTypeTag[P]]) = {
import c.universe._
val getTagTree = Select(Ident(newTermName("Macros")), "getTag")
val typeTags = c.Expr[List[ru.TypeTag[_]]](Apply(reify(List).tree, children.map( t =>
c.Expr(TypeApply(getTagTree, List(t.tree))).tree
).toList))
reify {
typeTags.splice.map { child =>
getBaseParam(child.tpe, baseTag.splice.tpe.typeSymbol, paramTag.splice.tpe.typeSymbol) -> child.tpe
}.toMap
}
}
def typeParamMap[B, P](children: B*)(implicit baseTag: ru.WeakTypeTag[B], paramTag: ru.WeakTypeTag[P]) = macro Macros.typeParamMapImpl[B, P]
}
However when I try to compile with an invocation of this, I get a huge compiler error: https://gist.github.com/4647812
Can anyone spot the problem, or suggest a better way of going about this?
Thanks!

Syntactic sugar for compile-time object creation in Scala

Lets say I have
trait fooTrait[T] {
def fooFn(x: T, y: T) : T
}
I want to enable users to quickly declare new instances of fooTrait with their own defined bodies for fooFn. Ideally, I'd want something like
val myFoo : fooTrait[T] = newFoo((x:T, y:T) => x+y)
to work. However, I can't just do
def newFoo[T](f: (x:T, y:T) => T) = new fooTrait[T] { def fooFn(x:T, y:T):T = f(x,y); }
because this uses closures, and so results in different objects when the program is run multiple times. What I really need is to be able to get the classOf of the object returned by newFoo and then have that be constructable on a different machine. What do I do?
If you're interested in the use case, I'm trying to write a Scala wrapper for Hadoop that allows you to execute
IO("Data") --> ((x: Int, y: Int) => (x, x+y)) --> IO("Out")
The thing in the middle needs to be turned into a class that implements a particular interface and can then be instantiated on different machines (executing the same jar file) from just the class name.
Note that Scala does the right thing with the syntactic sugar that converts (x:Int) => x+5 to an instance of Function1. My question is whether I can replicate this without hacking the Scala internals. If this was lisp (as I'm used to), this would be a trivial compile-time macro ... :sniff:
Here's a version that matches the syntax of what you list in the question and serializes/executes the anon-function. Note that this serializes the state of the Function2 object so that the serialized version can be restored on another machine. Just the classname is insufficient, as illustrated below the solution.
You should make your own encode/decode function, if even to just include your own Base64 implementation (not to rely on Sun's Hotspot).
object SHadoopImports {
import java.io._
implicit def functionToFooString[T](f:(T,T)=>T) = {
val baos = new ByteArrayOutputStream()
val oo = new ObjectOutputStream(baos)
oo.writeObject(f)
new sun.misc.BASE64Encoder().encode(baos.toByteArray())
}
implicit def stringToFun(s: String) = {
val decoder = new sun.misc.BASE64Decoder();
val bais = new ByteArrayInputStream(decoder.decodeBuffer(s))
val oi = new ObjectInputStream(bais)
val f = oi.readObject()
new {
def fun[T](x:T, y:T): T = f.asInstanceOf[Function2[T,T,T]](x,y)
}
}
}
// I don't really know what this is supposed to do
// just supporting the given syntax
case class IO(src: String) {
import SHadoopImports._
def -->(s: String) = new {
def -->(to: IO) = {
val IO(snk) = to
println("From: " + src)
println("Applying (4,5): " + s.fun(4,5))
println("To: " + snk)
}
}
}
object App extends Application {
import SHadoopImports._
IO("MySource") --> ((x:Int,y:Int)=>x+y) --> IO("MySink")
println
IO("Here") --> ((x:Int,y:Int)=>x*y+y) --> IO("There")
}
/*
From: MySource
Applying (4,5): 9
To: MySink
From: Here
Applying (4,5): 25
To: There
*/
To convince yourself that the classname is insufficient to use the function on another machine, consider the code below which creates 100 different functions. Count the classes on the filesystem and compare.
object App extends Application {
import SHadoopImports._
for (i <- 1 to 100) {
IO(i + ": source") --> ((x:Int,y:Int)=>(x*i)+y) --> IO("sink")
}
}
Quick suggestion: why don't you try to create an implicit def transforming FunctionN object to the trait expected by the --> method.
I do hope you won't have to use any macro for this!