I have a sum type, Mapping:
sealed trait Mapping
final case class XMapping(a:String)
final case class FallbackMapping(mappings: List[Mapping])
I have a typeclass defined as follows:
final case class Param(x:String)
trait ParameterLoader[T] {
def load(mapping:T) : List[Param]
}
With some instances:
object DefaultParameterLoaders {
implicit val xParameterLoader= new QueryParameterLoader[XMapping] {
override def load(mapping: XMapping): List[Param] = List(Param(mapping.a))
}
implicit val fallbackParameterLoader = new ParameterLoader[FallbackMapping] {
override def load(mapping: FallbackMapping): List[Param] =
mapping.mappings.flatMap(x => ???)
}
}
I can't find a way to have the implicit instances passed to the flatMap above. The error I get is that I'm missing an instance of ParameterLoader[Mapping]. Is there some way of telling the compiler that it should use whatever typeclass instances are in scope?
The type system is looking for a ParameterLoader[Mapping] specifically, meaning that a ParameterLoader[XMapping]/ParameterLoader[FallbackMapping] is not specific enough. You need to provide a ParameterLoader[Mapping]. You can do this using your existing definitions.
implicit def mappingLoader(implicit xpLoader: ParameterLoader[XMapping], fmLoader: ParameterLoader[FallbackMapping]) = new ParameterLoader[Mapping] {
def load(mapping: Mapping): List[QueryParam] =
mapping match {
case xm: XMapping = xpLoader.load(xm)
case fm: FallbackMapping => fmLoader.load(fm)
}
}
Alternatively, have your flatmap perform the matching logic:
implicit def fallbackParameterLoader(implicit xpLoader: ParameterLoader[XMapping]) = new ParameterLoader[FallbackMapping] {
override def load(mapping: FallbackMapping): List[Param] =
mapping.mappings.flatMap {
case xm: XMapping = xpLoader.load(xm)
case fm: FallbackMapping => this.load(fm)
}
}
Related
I have a lot of classes such as DataFrameFlow, TextFlow, RDDFlow. They all derive from base class Flow.
Now I want to write a function judgeFlow which can read from a path: String and return something representing exact Flow type from which I can create corresponding instance. The whole code seems like the following
def judgeFlow(path:String) = /*1*/ {
Flow.getStoreType(path) match {
case StoreType.tdw =>
DataFrameFlow
case StoreType.hdfs =>
TextFlow
}
}
def createFlow(typeInfo:/*2*/) = /*3*/{
new typeInfo()
}
However, I don't know how to write in place 1, 2 and 3.
EDIT
Knowing how to construct them is not enough here, because I also want the following:
pattern matching through typeInfo
some ways to do asInstanceOf
EDIT 2
Definition of Flow
abstract class Flow(var outputName: String) extends Serializable{
def this() = this("")
...
}
Definition of DataFrameFlow
class DataFrameFlow(d: DataFrame, path: String) extends Flow {
var data: DataFrame = d
def this(data: DataFrame) = this(data, "")
def this(path: String) = this(null, path)
def this() = this(null, "")
...
}
Pattern matching can't return different types from different cases. The type returned by pattern matching is the least upper bound of types returned in cases.
When someone wants to return different types, most probably he/she wants a type class.
sealed abstract class Flow
class DataFrameFlow extends Flow
class TextFlow extends Flow
class RDDFlow extends Flow
trait JudgeFlow[In] {
type Out <: Flow
def judgeFlow(in: In): Out
}
object JudgeFlow {
implicit val `case1`: JudgeFlow[???] { type Out = DataFrameFlow } = ???
implicit val `case2`: JudgeFlow[???] { type Out = TextFlow } = ???
implicit val `case3`: JudgeFlow[???] { type Out = RDDFlow } = ???
}
def judgeFlow[In](in: In)(implicit jf: JudgeFlow[In]): jf.Out = jf.judgeFlow(in)
But the trouble is that types are resolved at compile time. You seem to want to choose a case based on a value of string i.e. at runtime. So you can't return more specific types than just Flow at compile time.
flatMap with Shapeless yield FlatMapper not found
It's hard to guess your use case completely.
But using Scala reflection you can try
import scala.reflect.runtime.universe._
import scala.reflect.runtime.currentMirror
def judgeFlow(path:String): Type = {
Flow.getStoreType(path) match {
case StoreType.tdw =>
typeOf[DataFrameFlow]
case StoreType.hdfs =>
typeOf[TextFlow]
}
}
def createFlow(typeInfo: Type): Flow = {
val constructorSymbol = typeInfo.decl(termNames.CONSTRUCTOR).asMethod
val classSymbol = typeInfo.typeSymbol.asClass
val classMirror = currentMirror.reflectClass(classSymbol)
val constructorMirror = classMirror.reflectConstructor(constructorSymbol)
constructorMirror().asInstanceOf[Flow]
}
I want to have various "flavors" of a component, each that handles a different "wire" format (e.g. String, Byte array, etc.). Example below. The innards of the read() function aren't important.
Note that on use I need to cast parameter "Heavy" to thing.WIRE to work. Since this is my top-level API I don't want the users to have to cast. They've chosen the flavor when they call FantasticThing.apply (or accept the default). After that I'd rather a cast not be needed.
How can I avoid the cast and have Scala realize that read() argument is a String based on StringFlavor being chosen?
trait Flavor {
type WIRE
def read[T](wire: WIRE)(implicit tt: TypeTag[T]): T
}
trait Maker {
def make(): Flavor
}
object StringFlavor extends Maker {
def make(): Flavor { type WIRE = String } = StringFlavor()
}
case class StringFlavor() extends Flavor {
type WIRE = String
def read[T](wire: String)(implicit tt: TypeTag[T]): T = {
println(tt.tpe)
if(tt.tpe =:= typeOf[Int]) {
5.asInstanceOf[T]
} else
throw new Exception("Boom")
}
}
object FantasticThing {
def apply[WIRE](maker: Maker = StringFlavor): Flavor = maker.make()
}
object RunMe extends App {
val thing: Flavor = FantasticThing(StringMaker)
println(thing.read[Int]("Heavy".asInstanceOf[thing.WIRE])) // <-- How can I avoid this cast?
}
Edit based on Luis Miguel's note: I can't really add that type to FantasticThing.apply() or I'd lose pluggability. I want users to easily select the Flavor they want. I've refactored a bit to show this with a factory pattern, which does add the type info you suggested, but still unfortunately leaves me with a need to cast top-level.
If I provide a bunch of Flavors then users should be able to do something like:
val foo = FantasticThing(ByteArrayFlavor)
You can make WIRE a type parameter and propagate it through a type member or your Maker type. I.e:
import scala.reflect.runtime.universe._
trait Flavor[WIRE] {
def read[T](wire: WIRE)(implicit tt: TypeTag[T]): T
}
trait Maker {
type O
def make(): Flavor[O]
}
object StringMaker extends Maker {
type O = String
def make(): Flavor[O] = StringFlavor()
}
case class StringFlavor() extends Flavor[String] {
def read[T](wire: String)(implicit tt: TypeTag[T]): T = {
if(tt.tpe =:= typeOf[Int]) {
5.asInstanceOf[T]
} else
throw new Exception("Boom")
}
}
object FantasticThing {
def apply(): Flavor[String] = StringMaker.make()
def apply(maker: Maker): Flavor[maker.O] = maker.make() // Path dependent type.
}
object RunMe extends App {
val thing: Flavor[String] = FantasticThing(StringMaker)
thing.read[Int]("Heavy") // res0: Int = 5
}
Edit: Added no-arg apply() to this anwser. If a default value for maker is used (e.g. StringMaker) you get a compile error because argument "Heavy" is now supposed to be type Maker.O. Adding the no-arg apply solves this problem while providing the same experience to the caller.
I took the liberty to modify your code with the intention to show how (what I understand of) your problem can be solved using typeclasses and type parameters, instead of type members.
import scala.reflect.runtime.universe.{TypeTag, typeOf}
implicit class Json(val underlying: String) extends AnyVal
implicit class Csv(val underlying: String) extends AnyVal
trait Flavor[W] {
def read[T](wire: W)(implicit tt: TypeTag[T]): T
}
trait Maker[W] {
def make(): Flavor[W]
}
object Maker {
implicit val StringFlavorMaker: Maker[String] = new Maker[String] {
override def make(): Flavor[String] = StringFlavor
}
implicit val JsonFlavorMaker: Maker[Json] = new Maker[Json] {
override def make(): Flavor[Json] = JsonFlavor
}
implicit val CsvFlavorMaker: Maker[Csv] = new Maker[Csv] {
override def make(): Flavor[Csv] = CsvFlavor
}
}
case object StringFlavor extends Flavor[String] {
override final def read[T](wire: String)(implicit tt: TypeTag[T]): T = {
if(tt.tpe =:= typeOf[Int])
0.asInstanceOf[T]
else
throw new Exception("Boom 1")
}
}
case object JsonFlavor extends Flavor[Json] {
override final def read[T](wire: Json)(implicit tt: TypeTag[T]): T = {
if(tt.tpe =:= typeOf[Int])
3.asInstanceOf[T]
else
throw new Exception("Boom 2")
}
}
case object CsvFlavor extends Flavor[Csv] {
override final def read[T](wire: Csv)(implicit tt: TypeTag[T]): T = {
if(tt.tpe =:= typeOf[Int])
5.asInstanceOf[T]
else
throw new Exception("Boom 3")
}
}
object FantasticThing {
def apply[W](implicit maker: Maker[W]): Flavor[W] = maker.make()
}
Then you can create and user any flavour (given there is an implicit maker in scope) this way.
val stringFlavor = FantasticThing[String]
// stringFlavor: Flavor[String] = StringFlavor
stringFlavor.read[Int]("Heavy")
// res0: Int = 0
val jsonFlavor = FantasticThing[Json]
// jsonFlavor: Flavor[Json] = JsonFlavor
jsonFlavor.read[Int]("{'heavy':'true'}")
// res1: Int = 3
val csvFlavor = FantasticThing[Csv]
// csvFlavor: Flavor[Csv] = CsvFlavor
csvFlavor.read[Int]("Hea,vy")
// res2: Int = 0
In general, is better to stay off of type members, since they are more complex and used for more advanced stuff like path dependent types.
Let me know in the comments if you have any doubt.
DISCLAIMER: I am bad with type members (still learning about them), that may motivate me to use different alternatives. - In any case, I hope you can apply something similar to your real problem..
I have written automatic type class derivation in order to automatically generate elasticsearch Json mapping for case classes.
For that I am using the TypeClass type class in shapeless.
The problem I have is that many fields in the case classes we use are Scala enumerations.
For example
object ConnectionState extends Enumeration {
type ConnectionState = Value
val ordering, requested, pending, available, down, deleting, deleted, rejected = Value
}
Or
object ProductCodeType extends Enumeration {
type ProductCodeType = Value
val devpay, marketplace = Value
}
It seems I have to define a specific implicit instance for every Enumeration that is defined in order that the automatic derivation will pick it up (e.g. for both ConnectionState and ProductCodeType).
I cannot have one implicit def for Enumeration such as
implicit def enumerationMapping: MappingEncoder[Enumeration] = new MappingEncoder[Enumeration] {
def toMapping = jSingleObject("type", jString("text"))
}
that will work for all enumeration types.
I tried making the type class covariant, and a bunch of other things but nothing helped.
Any ideas?
Here is the derivation code:
object Mapping {
trait MappingEncoder[T] {
def toMapping: Json
}
object MappingEncoder extends LabelledProductTypeClassCompanion[MappingEncoder] {
implicit val stringMapping: MappingEncoder[String] = new MappingEncoder[String] {
def toMapping = jSingleObject("type", jString("text"))
}
implicit val intMapping: MappingEncoder[Int] = new MappingEncoder[Int] {
def toMapping = jSingleObject("type", jString("integer"))
}
implicit def seqMapping[T: MappingEncoder]: MappingEncoder[Seq[T]] = new MappingEncoder[Seq[T]] {
def toMapping = implicitly[MappingEncoder[T]].toMapping
}
implicit def optionMapping[T: MappingEncoder]: MappingEncoder[Option[T]] = new MappingEncoder[Option[T]] {
def toMapping = implicitly[MappingEncoder[T]].toMapping
}
object typeClass extends LabelledProductTypeClass[MappingEncoder] {
def emptyProduct = new MappingEncoder[HNil] {
def toMapping = jEmptyObject
}
def product[F, T <: HList](name: String, sh: MappingEncoder[F], st: MappingEncoder[T]) = new MappingEncoder[F :: T] {
def toMapping = {
val head = sh.toMapping
val tail = st.toMapping
(name := head) ->: tail
}
}
def project[F, G](instance: => MappingEncoder[G], to: F => G, from: G => F) = new MappingEncoder[F] {
def toMapping = jSingleObject("properties", instance.toMapping)
}
}
}
}
I was able to fix the problem by adding additional implicit defs to scope:
implicit def enumerationMapping[T <: Enumeration#Value]: MappingEncoder[T] = new MappingEncoder[T] {
def toMapping = jSingleObject("type", jString("text"))
}
implicit def enumerationSeqMapping[T <: Enumeration#Value]: MappingEncoder[Seq[T]] = new MappingEncoder[Seq[T]] {
def toMapping = jSingleObject("type", jString("text"))
}
The second implicit was needed as some of the case classes had members of type Seq[T] where T is some enumeration type.
Just when I thought I understood the basics of Scala's type system... :/
I'm trying to implement a class that reads the contents of a file and outputs a set of records. A record might be a single line, but it could also be a block of bytes, or anything. So what I'm after is a structure that allows the type of Reader to imply the type of the Record, which in turn will imply the correct Parser to use.
This structure works as long as MainApp.records(f) only returns one type of Reader. As soon as it can return more, I get this error:
could not find implicit value for parameter parser
I think the problem lies with the typed trait definitions at the top, but I cannot figure out how to fix the issue...
// Core traits
trait Record[T]
trait Reader[T] extends Iterable[Record[T]]
trait Parser[T] {
def parse(r: Record[T]): Option[Int]
}
// Concrete implementations
class LineRecord[T] extends Record[T]
class FileReader[T](f:File) extends Reader[T] {
val lines = Source.fromFile(f).getLines()
def iterator: Iterator[LineRecord[T]] =
new Iterator[LineRecord[T]] {
def next() = new LineRecord[T]
def hasNext = lines.hasNext
}
}
trait TypeA
object TypeA {
implicit object TypeAParser extends Parser[TypeA] {
def parse(r: Record[TypeA]): Option[Int] = ???
}
}
trait TypeB
object TypeB {
implicit object TypeBParser extends Parser[TypeB] {
def parse(r: Record[TypeB]): Option[Int] = ???
}
}
// The "app"
object MainApp {
def process(f: File) =
records(f) foreach { r => parse(r) }
def records(f: File) = {
if(true)
new FileReader[TypeA](f)
else
new FileReader[TypeB](f)
}
def parse[T](r: Record[T])(implicit parser: Parser[T]): Option[Int] =
parser.parse(r)
}
First off you must import the implicit object in order to use them:
import TypeA._
import TypeB._
That's not enough though. It seems like you're trying to apply implicits dynamically. That's not possible; they have to be found compile time.
If you import the objects as above and change the records so that the compiler finds the correct generic it will run fine:
def records(f: File) = new FileReader[TypeA](f)
But then it may not be what you were looking for ;)
The problem is that the return type of your records method is basically FileReader[_] (since it can return either FileReader[TypeA] or FileReader[TypeB]), and you don't provide an implicit argument of type Parser[Any]. If you remove the if-expression the return type is inferred to FileReader[TypeA], which works fine. I'm not sure what you're trying to do, but obviously the compiler can't select implicit argument based upon a type that is only known at runtime.
1) Using type with implicit inside as type parameter - doesn't bind this implicit to the host type, to do this change objects to the traits and mix them instead of generalizing (type-parametrizing):
def records(f: File) = {
if(true)
new FileReader(f) with TypeA
else
new FileReader(f) with TypeB
}
2) The parser should be in scope of function that calls parse. So you may try smthg like that:
def process(f: File) = {
val reader = records(f);
import reader._
reader foreach { r => parse(r) }
}
PlanB) Simpler alternative is to define type-parameter specific implicit methods inside the AppMain (or some trait mixed in), but it will work only if TypeA/TypeB is known on compile time, so records method can return concrete type:
implicit class TypeAParser(r: Record[TypeA]) {
def parse: Option[Int] = ???
}
implicit class TypeBParser(r: Record[TypeB]) {
def parse: Option[Int] = ???
}
def process[T <: TypeAorB](f: File) =
records[T](f).foreach(_.parse)
def recordsA[T <: TypeAorB](f: File) = new FileReader[T](f)
Here is, I think, the full set of modifications you need to do to get where I think you want to go.
import scala.io.Source
import java.io.File
import reflect.runtime.universe._
// Core traits
trait Record[+T]
trait Reader[+T] extends Iterable[Record[T]]
trait Parser[-T] {
def parse(r: Record[T]): Option[Int]
}
// Concrete implementations [unmodified]
class LineRecord[T] extends Record[T]
class FileReader[T](f:File) extends Reader[T] {
val lines = Source.fromFile(f).getLines()
def iterator: Iterator[LineRecord[T]] =
new Iterator[LineRecord[T]] {
def next() = new LineRecord[T]
def hasNext = lines.hasNext
}
}
sealed trait Alternatives
case class TypeA() extends Alternatives
object TypeA {
implicit object TypeAParser extends Parser[TypeA] {
def parse(r: Record[TypeA]): Option[Int] = ???
}
}
case class TypeB() extends Alternatives
object TypeB {
implicit object TypeBParser extends Parser[TypeB] {
def parse(r: Record[TypeB]): Option[Int] = ???
}
}
class ParseAlternator(parserA: Parser[TypeA], parserB: Parser[TypeB]) extends Parser[Alternatives] {
def parse(r: Record[Alternatives]): Option[Int] = r match {
case x: Record[TypeA #unchecked] if typeOf[Alternatives] =:= typeOf[TypeA] => parserA.parse(x)
case x: Record[TypeB #unchecked] if typeOf[Alternatives] =:= typeOf[TypeB] => parserB.parse(x)
}
}
object ParseAlternator {
implicit def parseAlternator(implicit parserA: Parser[TypeA], parserB: Parser[TypeB]): Parser[Alternatives] = new ParseAlternator(parserA, parserB)
}
// The "app"
object MainApp {
import ParseAlternator._
def process(f: File) =
records(f) foreach { r => parse(r) }
def records(f: File): Reader[Alternatives] = {
if(true)
new FileReader[TypeA](f)
else
new FileReader[TypeB](f)
}
def parse[T](r: Record[T])(implicit parser: Parser[T]): Option[Int] =
parser.parse(r)
}
The gist of it is: all of this would be completely classsical if only your parse instance did not have to pattern-match on a generic type but dealt directly with an Alternative instead.
It's this limitation (inherited from the JVM) that scala can't properly pattern-match on an object of a parametric type that requires the reflection & typeOf usage. Without it, you would just have type alternatives for your content (TypeA, TypeB), which you would add to a sealed trait, and which you would dispatch on, in an implicit that produces a Parser for their supertype.
Of course this isn't the only solution, it's just what I think is the meeting point of what's closest to what you're trying to do, with what's most idiomatic.
I am trying to write a generic method f[T](id:String) that is something like this:
case class A(x:String)
case class B(y:String)
case class C(z:String)
def f[T](id:String): T = { /* equivalent to T(id) */ }
val result1:A = f[A]("123") // returns A("123")
val result2:B = f{B]("345") // returns B("345")
val result3:C = f[C]("567") // returns C("567")
Unfortunately I cannot figure out how to work with the type T inside the method, besides using reflection. By "working with the type T" i mean for example being able to do something like the following, which I know doesn't work (for illustration purposes only):
T match {
case A => A(id)
case B => B(id)
}
or simply invoke T(ID) to create a new object of whatever type T is.
I can of course break up this into three methods:
def f1(id:String): A = { A(id) }
def f2(id:String): B = { B(id) }
def f3(id:String): C = { C(id) }
val result1:A = f1("123") // returns A("123")
val result2:B = f2("345") // returns B("345")
val result3:C = f3("567") // returns C("567")
but I'm hoping there is a way to keep it as one generic method to avoid some ugly boilerplate code duplication, and still be nearl as fast as the tree method version.
If you do not want to use reflection (ClassTag or TypeTag), you could use a Factory type class to achieve the desired functionality (unless it defeats the purpose of your generic function by generating a lot of duplicated simple code ;)).
case class A(s: String)
case class B(s: String)
case class C(s: String)
trait Factory[T] extends ((String) => T) {
def apply(arg: String): T
}
object Factory {
implicit object AFactory extends Factory[A] {
override def apply(arg: String): A = A(arg)
}
implicit object BFactory extends Factory[B] {
override def apply(arg: String): B = B(arg)
}
implicit object CFactory extends Factory[C] {
override def apply(arg: String): C = C(arg)
}
}
def create[T : Factory](arg: String): T = implicitly[Factory[T]].apply(arg)
create[A]("foo") | -> res0: A = A(foo)