Is there some way in scala that I can return a type? - scala

I have a lot of classes such as DataFrameFlow, TextFlow, RDDFlow. They all derive from base class Flow.
Now I want to write a function judgeFlow which can read from a path: String and return something representing exact Flow type from which I can create corresponding instance. The whole code seems like the following
def judgeFlow(path:String) = /*1*/ {
Flow.getStoreType(path) match {
case StoreType.tdw =>
DataFrameFlow
case StoreType.hdfs =>
TextFlow
}
}
def createFlow(typeInfo:/*2*/) = /*3*/{
new typeInfo()
}
However, I don't know how to write in place 1, 2 and 3.
EDIT
Knowing how to construct them is not enough here, because I also want the following:
pattern matching through typeInfo
some ways to do asInstanceOf
EDIT 2
Definition of Flow
abstract class Flow(var outputName: String) extends Serializable{
def this() = this("")
...
}
Definition of DataFrameFlow
class DataFrameFlow(d: DataFrame, path: String) extends Flow {
var data: DataFrame = d
def this(data: DataFrame) = this(data, "")
def this(path: String) = this(null, path)
def this() = this(null, "")
...
}

Pattern matching can't return different types from different cases. The type returned by pattern matching is the least upper bound of types returned in cases.
When someone wants to return different types, most probably he/she wants a type class.
sealed abstract class Flow
class DataFrameFlow extends Flow
class TextFlow extends Flow
class RDDFlow extends Flow
trait JudgeFlow[In] {
type Out <: Flow
def judgeFlow(in: In): Out
}
object JudgeFlow {
implicit val `case1`: JudgeFlow[???] { type Out = DataFrameFlow } = ???
implicit val `case2`: JudgeFlow[???] { type Out = TextFlow } = ???
implicit val `case3`: JudgeFlow[???] { type Out = RDDFlow } = ???
}
def judgeFlow[In](in: In)(implicit jf: JudgeFlow[In]): jf.Out = jf.judgeFlow(in)
But the trouble is that types are resolved at compile time. You seem to want to choose a case based on a value of string i.e. at runtime. So you can't return more specific types than just Flow at compile time.
flatMap with Shapeless yield FlatMapper not found
It's hard to guess your use case completely.
But using Scala reflection you can try
import scala.reflect.runtime.universe._
import scala.reflect.runtime.currentMirror
def judgeFlow(path:String): Type = {
Flow.getStoreType(path) match {
case StoreType.tdw =>
typeOf[DataFrameFlow]
case StoreType.hdfs =>
typeOf[TextFlow]
}
}
def createFlow(typeInfo: Type): Flow = {
val constructorSymbol = typeInfo.decl(termNames.CONSTRUCTOR).asMethod
val classSymbol = typeInfo.typeSymbol.asClass
val classMirror = currentMirror.reflectClass(classSymbol)
val constructorMirror = classMirror.reflectConstructor(constructorSymbol)
constructorMirror().asInstanceOf[Flow]
}

Related

What's the best way to store different types in a collection

I have a generic class:
class GenericType[T] {
def func(t: T) = ???
}
I need to implement a function that takes a List[String] and outputs the corresponding GenericType[T]. For example, if a client passes in List("SomeType1", "SomeType2"), the function should return List(GenericType[SomeType1], GenericType[SomeType2]). Basically, there's a string that maps to a type.
I don't find a good way to represent the return type for such function. Seq[GenericType[_]] as the return type can be an option but it requires the client to cast it into corresponding subclasses to invoke func as the type info is lost.
Alternatively, a case class can be used but this is not flexible as I need to modify the case class every time a new subclass is added.
case class (s1: Option[GenericType[SomeType1]] = None, s2: Option[SomeType2] = None, ...)
I'm curious what's a good way to represent the return type?
The easiest is
def stringToGenericType(s: String): GenericType[_] = s match {
case "SomeType1" => new GenericType[SomeType1]
case "SomeType2" => new GenericType[SomeType2]
}
GenericType[_] (or GenericType[Any] if you make GenericType covariant: class GenericType[+T]) would be an honest return type. You can't know at compile time which specific GenericType[...] you return based on a runtime string. Anyway you can't avoid casting somewhere because you can't guarantee the type statically. Anyway this kind of programming can't be type-safe.
Since we're doing a choice based on a runtime string, compile-time techniques (macros, implicits, Shapeless) are off the table. (*)
Actually, currently you don't need even runtime reflection. Your class GenericType[T] and method func in it seem not to do anything at runtime differently depending on type T. GenericType[SomeType1] and GenericType[SomeType2] are just GenericType[_] at runtime. So even the following implementation is possible
def stringToGenericType(s: String): GenericType[_] = new GenericType[Any]
Another situation would be if you created instances of different classes
class GenericType1[T]
class GenericType2[T]
import scala.reflect.runtime
import scala.reflect.runtime.universe._
val runtimeMirror = runtime.currentMirror
def stringToGenericTypeX(s: String): Any = {
val classSymbol = runtimeMirror.staticClass(s)
val constructorSymbol = classSymbol.typeSignature.decl(termNames.CONSTRUCTOR).asMethod
runtimeMirror.reflectClass(classSymbol).reflectConstructor(constructorSymbol).apply()
}
or you called different methods
class GenericType[T] {
def func1(t: T) = ???
def func2(t: T) = ???
}
def callFuncX(methodName: String, t: Any) = {
val classSymbol = runtimeMirror.classSymbol(classOf[GenericType[_]])
val methodSymbol = classSymbol.typeSignature.decl(TermName(methodName)).asMethod
runtimeMirror.reflect(new GenericType[Any]).reflectMethod(methodSymbol).apply(t)
}
or something behaved at runtime differently depending on type T
class GenericType[T: ClassTag] {
def func(t: T) = println(classTag[T].runtimeClass)
}
import scala.tools.reflect.ToolBox
val toolbox = runtimeMirror.mkToolBox()
def stringToGenericType(s: String): GenericType[_] = {
toolbox.eval(q"new GenericType[${TypeName(s)}]").asInstanceOf[GenericType[_]]
}
(*) Well, actually the first pattern matching (as I wrote it) can be automated with a macro
// in a different subproject
import scala.language.experimental.macros
import scala.reflect.macros.blackbox
def stringToGenericType(s: String): GenericType[_] = macro stringToGenericTypeImpl
def stringToGenericTypeImpl(c: blackbox.Context)(s: c.Tree): c.Tree = {
import c.universe._
val cases = List("SomeType1", "SomeType2").map(str =>
cq"$str => new GenericType[${TypeName(str)}]"
)
q"$s match { case ..$cases }"
}
val s = "SomeTime1"
stringToGenericType(s)
// scalacOptions += "-Ymacro-debug-lite"
//s match {
// case "SomeType1" => new GenericType[SomeType1]()
// case "SomeType2" => new GenericType[SomeType2]()
//}

Accessing implicit parameters explicitly at runtime

I'm trying to access an implicit parameter for a generic type. Scala is able to find the implicit just fine in the straightforward case by calling a method with an explicit generic type, such as with printGenericType[Person] below.
However, if I create a TypeTag[Person] and pass it to printGenericTypeGivenTypeTag, Scala is unable to find the implicit parameter.
case class Person(name: String)
case class Animal(age: Int)
implicit val p = Person("cory")
implicit val a = Animal(2)
def main(args: Array[String]): Unit = {
// Can find the implicit Person, prints "Hello Person(cory)"
printGenericType[Person]
// Can find the implicit Animal, prints "Hello Animal(2)"
printGenericType[Animal]
// See comment below
printNamedType("Person")
printNamedType("Animal")
}
def printGenericType[T](implicit t: T) = {
println(s"Hello $t")
}
def printGenericTypeGivenTypeTag[T](typeTag: TypeTag[T])(implicit t: T) = {
println(s"Hello $t")
}
def printNamedType[T](name: String) = {
val typetag: TypeTag[T] = getTypeTag[T](name)
// Cannot find the implicit of type T, compiler error
printGenericTypeGivenTypeTag(typetag)
}
def getTypeTag[T](name:String): TypeTag[T] = ... //Implementation irrelevant
If I understand correctly, Scala locates implicit parameters at compile time, so it makes sense that it can't find an implicit parameter for the generic type T at compile time.
However, I know that an implicit instance of T does exist. Is it possible to rewrite printGenericTypeGivenTypeTag in such a way as to find the implicit value for T? At runtime, the method has access to the actual type of T, so it seems it should be able to locate an implicit parameter of the same type that is in scope.
For the curious, the reasoning behind this is to avoid this:
name match {
case "Person" => printGenericType[Person]
case "Animal" => printGenericType[Animal]
}
To answer the question
You're not really wanting to pass the T implicitly, but rather the TypeTag, and not the T. Here's what I mean, and you're probably better off with an implicit value class.
implicit class GenericPrinter[T](val obj: T) extends AnyVal {
def printGenericType()(implicit tag: TypeTag[T]) = {
// do stuff with the tag
Console.println("Hello $obj")
}
}
val x: Person = Person(...)
x.printGenericType
Now solving the real problem
If you are trying to print case classes, I'd probably go down the implicit macro route for added convenience. It's really trivial to write up a macro that does this for us, e.g output a debug string based on all the constructor params of an arbitrary case class.
trait DeepPrinter[T <: Product with Serializable] {
/**
* Prints a deeply nested debug string for a given case class.
* This uses implicit macros to materialise the printer type class.
* In English, when we request for an implicit printer: DeepPrinter[T],
* the pre-defined compile time macro will generate this method for us
* based on the fields of the given case class.
*
* #param sep A separator to use to delimit the rows in a case class.
* #return A fully traced debug string so we can see how a case class looks like.
*/
def debugString(sep: String = "\n"): String
}
object DeepPrinter {
implicit def deepPrinter[T <: Product with Serializable] = macro DeepPrinterImpl.deepPrinterImpl[T]
}
And the macro is pretty trivial, looks kind of like this.
import language.experimental.macros
import scala.reflect.macros.blackbox
#macrocompat.bundle
class DeepPrinterImpl(val c: blackbox.Context) {
import c.universe._
object CaseField {
def unapply(symbol: TermSymbol): Option[(Name, Type)] = {
if (symbol.isVal && symbol.isCaseAccessor) {
Some(symbol.name -> symbol.typeSignature)
} else {
None
}
}
}
def fields(tpe: Type): Iterable[(Name, Type)] = {
tpe.decls.collect { case CaseField(nm, tpeSn) => nm -> tpeSn }
}
def materialize[T : c.WeakTypeTag]: c.Expr[DeepPrinter[T]] = {
val tpe = weakTypeOf[T]
val (names, types) = fields(tpe).unzip
// change the package name to the correct one here!
val tree = q"""
new com.bla.bla.DeepPrinter[$tpe] {
def debugString(sep: String = "\n") = Seq(..$names).mkString(sep)
}
"""
c.Expr[DeepPrinter[T]](tree)
}
}

Scala Typeclass for composites

I have a sum type, Mapping:
sealed trait Mapping
final case class XMapping(a:String)
final case class FallbackMapping(mappings: List[Mapping])
I have a typeclass defined as follows:
final case class Param(x:String)
trait ParameterLoader[T] {
def load(mapping:T) : List[Param]
}
With some instances:
object DefaultParameterLoaders {
implicit val xParameterLoader= new QueryParameterLoader[XMapping] {
override def load(mapping: XMapping): List[Param] = List(Param(mapping.a))
}
implicit val fallbackParameterLoader = new ParameterLoader[FallbackMapping] {
override def load(mapping: FallbackMapping): List[Param] =
mapping.mappings.flatMap(x => ???)
}
}
I can't find a way to have the implicit instances passed to the flatMap above. The error I get is that I'm missing an instance of ParameterLoader[Mapping]. Is there some way of telling the compiler that it should use whatever typeclass instances are in scope?
The type system is looking for a ParameterLoader[Mapping] specifically, meaning that a ParameterLoader[XMapping]/ParameterLoader[FallbackMapping] is not specific enough. You need to provide a ParameterLoader[Mapping]. You can do this using your existing definitions.
implicit def mappingLoader(implicit xpLoader: ParameterLoader[XMapping], fmLoader: ParameterLoader[FallbackMapping]) = new ParameterLoader[Mapping] {
def load(mapping: Mapping): List[QueryParam] =
mapping match {
case xm: XMapping = xpLoader.load(xm)
case fm: FallbackMapping => fmLoader.load(fm)
}
}
Alternatively, have your flatmap perform the matching logic:
implicit def fallbackParameterLoader(implicit xpLoader: ParameterLoader[XMapping]) = new ParameterLoader[FallbackMapping] {
override def load(mapping: FallbackMapping): List[Param] =
mapping.mappings.flatMap {
case xm: XMapping = xpLoader.load(xm)
case fm: FallbackMapping => this.load(fm)
}
}

Could not find implicit value for parameter x

Just when I thought I understood the basics of Scala's type system... :/
I'm trying to implement a class that reads the contents of a file and outputs a set of records. A record might be a single line, but it could also be a block of bytes, or anything. So what I'm after is a structure that allows the type of Reader to imply the type of the Record, which in turn will imply the correct Parser to use.
This structure works as long as MainApp.records(f) only returns one type of Reader. As soon as it can return more, I get this error:
could not find implicit value for parameter parser
I think the problem lies with the typed trait definitions at the top, but I cannot figure out how to fix the issue...
// Core traits
trait Record[T]
trait Reader[T] extends Iterable[Record[T]]
trait Parser[T] {
def parse(r: Record[T]): Option[Int]
}
// Concrete implementations
class LineRecord[T] extends Record[T]
class FileReader[T](f:File) extends Reader[T] {
val lines = Source.fromFile(f).getLines()
def iterator: Iterator[LineRecord[T]] =
new Iterator[LineRecord[T]] {
def next() = new LineRecord[T]
def hasNext = lines.hasNext
}
}
trait TypeA
object TypeA {
implicit object TypeAParser extends Parser[TypeA] {
def parse(r: Record[TypeA]): Option[Int] = ???
}
}
trait TypeB
object TypeB {
implicit object TypeBParser extends Parser[TypeB] {
def parse(r: Record[TypeB]): Option[Int] = ???
}
}
// The "app"
object MainApp {
def process(f: File) =
records(f) foreach { r => parse(r) }
def records(f: File) = {
if(true)
new FileReader[TypeA](f)
else
new FileReader[TypeB](f)
}
def parse[T](r: Record[T])(implicit parser: Parser[T]): Option[Int] =
parser.parse(r)
}
First off you must import the implicit object in order to use them:
import TypeA._
import TypeB._
That's not enough though. It seems like you're trying to apply implicits dynamically. That's not possible; they have to be found compile time.
If you import the objects as above and change the records so that the compiler finds the correct generic it will run fine:
def records(f: File) = new FileReader[TypeA](f)
But then it may not be what you were looking for ;)
The problem is that the return type of your records method is basically FileReader[_] (since it can return either FileReader[TypeA] or FileReader[TypeB]), and you don't provide an implicit argument of type Parser[Any]. If you remove the if-expression the return type is inferred to FileReader[TypeA], which works fine. I'm not sure what you're trying to do, but obviously the compiler can't select implicit argument based upon a type that is only known at runtime.
1) Using type with implicit inside as type parameter - doesn't bind this implicit to the host type, to do this change objects to the traits and mix them instead of generalizing (type-parametrizing):
def records(f: File) = {
if(true)
new FileReader(f) with TypeA
else
new FileReader(f) with TypeB
}
2) The parser should be in scope of function that calls parse. So you may try smthg like that:
def process(f: File) = {
val reader = records(f);
import reader._
reader foreach { r => parse(r) }
}
PlanB) Simpler alternative is to define type-parameter specific implicit methods inside the AppMain (or some trait mixed in), but it will work only if TypeA/TypeB is known on compile time, so records method can return concrete type:
implicit class TypeAParser(r: Record[TypeA]) {
def parse: Option[Int] = ???
}
implicit class TypeBParser(r: Record[TypeB]) {
def parse: Option[Int] = ???
}
def process[T <: TypeAorB](f: File) =
records[T](f).foreach(_.parse)
def recordsA[T <: TypeAorB](f: File) = new FileReader[T](f)
Here is, I think, the full set of modifications you need to do to get where I think you want to go.
import scala.io.Source
import java.io.File
import reflect.runtime.universe._
// Core traits
trait Record[+T]
trait Reader[+T] extends Iterable[Record[T]]
trait Parser[-T] {
def parse(r: Record[T]): Option[Int]
}
// Concrete implementations [unmodified]
class LineRecord[T] extends Record[T]
class FileReader[T](f:File) extends Reader[T] {
val lines = Source.fromFile(f).getLines()
def iterator: Iterator[LineRecord[T]] =
new Iterator[LineRecord[T]] {
def next() = new LineRecord[T]
def hasNext = lines.hasNext
}
}
sealed trait Alternatives
case class TypeA() extends Alternatives
object TypeA {
implicit object TypeAParser extends Parser[TypeA] {
def parse(r: Record[TypeA]): Option[Int] = ???
}
}
case class TypeB() extends Alternatives
object TypeB {
implicit object TypeBParser extends Parser[TypeB] {
def parse(r: Record[TypeB]): Option[Int] = ???
}
}
class ParseAlternator(parserA: Parser[TypeA], parserB: Parser[TypeB]) extends Parser[Alternatives] {
def parse(r: Record[Alternatives]): Option[Int] = r match {
case x: Record[TypeA #unchecked] if typeOf[Alternatives] =:= typeOf[TypeA] => parserA.parse(x)
case x: Record[TypeB #unchecked] if typeOf[Alternatives] =:= typeOf[TypeB] => parserB.parse(x)
}
}
object ParseAlternator {
implicit def parseAlternator(implicit parserA: Parser[TypeA], parserB: Parser[TypeB]): Parser[Alternatives] = new ParseAlternator(parserA, parserB)
}
// The "app"
object MainApp {
import ParseAlternator._
def process(f: File) =
records(f) foreach { r => parse(r) }
def records(f: File): Reader[Alternatives] = {
if(true)
new FileReader[TypeA](f)
else
new FileReader[TypeB](f)
}
def parse[T](r: Record[T])(implicit parser: Parser[T]): Option[Int] =
parser.parse(r)
}
The gist of it is: all of this would be completely classsical if only your parse instance did not have to pattern-match on a generic type but dealt directly with an Alternative instead.
It's this limitation (inherited from the JVM) that scala can't properly pattern-match on an object of a parametric type that requires the reflection & typeOf usage. Without it, you would just have type alternatives for your content (TypeA, TypeB), which you would add to a sealed trait, and which you would dispatch on, in an implicit that produces a Parser for their supertype.
Of course this isn't the only solution, it's just what I think is the meeting point of what's closest to what you're trying to do, with what's most idiomatic.

Method polymorphism

I am trying to write a generic method f[T](id:String) that is something like this:
case class A(x:String)
case class B(y:String)
case class C(z:String)
def f[T](id:String): T = { /* equivalent to T(id) */ }
val result1:A = f[A]("123") // returns A("123")
val result2:B = f{B]("345") // returns B("345")
val result3:C = f[C]("567") // returns C("567")
Unfortunately I cannot figure out how to work with the type T inside the method, besides using reflection. By "working with the type T" i mean for example being able to do something like the following, which I know doesn't work (for illustration purposes only):
T match {
case A => A(id)
case B => B(id)
}
or simply invoke T(ID) to create a new object of whatever type T is.
I can of course break up this into three methods:
def f1(id:String): A = { A(id) }
def f2(id:String): B = { B(id) }
def f3(id:String): C = { C(id) }
val result1:A = f1("123") // returns A("123")
val result2:B = f2("345") // returns B("345")
val result3:C = f3("567") // returns C("567")
but I'm hoping there is a way to keep it as one generic method to avoid some ugly boilerplate code duplication, and still be nearl as fast as the tree method version.
If you do not want to use reflection (ClassTag or TypeTag), you could use a Factory type class to achieve the desired functionality (unless it defeats the purpose of your generic function by generating a lot of duplicated simple code ;)).
case class A(s: String)
case class B(s: String)
case class C(s: String)
trait Factory[T] extends ((String) => T) {
def apply(arg: String): T
}
object Factory {
implicit object AFactory extends Factory[A] {
override def apply(arg: String): A = A(arg)
}
implicit object BFactory extends Factory[B] {
override def apply(arg: String): B = B(arg)
}
implicit object CFactory extends Factory[C] {
override def apply(arg: String): C = C(arg)
}
}
def create[T : Factory](arg: String): T = implicitly[Factory[T]].apply(arg)
create[A]("foo") | -> res0: A = A(foo)