Assuming you have the following code
trait T {
}
case class First(int:Int) extends T
case class Second(int:Int) extends T
val a:Option[T] = Option(First(3))
val b:Option[Second] = None
def test[A](a:Option[A])(implicit manifest:Manifest[Option[A]]) = {
a match {
case Some(z) => println("we have a value here")
case None => a match {
case t:Option[First] => println("instance of first class")
case s:Option[Second] => println("instance of second class")
}
}
}
test(b)
How would you extract what type the enclosing A is if the option happens to be a None. I have attempted to do various combinations of manifests, but none of them seem to work, each time it complains about the types being eliminated with erasure, i.e.
non-variable type argument First in type pattern Option[First] is unchecked since it is eliminated by erasure
You can use a type tag, a modern replacement for Manifest:
import scala.reflect.runtime.universe._
trait T
case class First(int:Int) extends T
case class Second(int:Int) extends T
def test[A: TypeTag](a: Option[A]) = {
a match {
case Some(_) => println("we have a value here")
case None => typeOf[A] match {
case t if t =:= typeOf[First] => println("instance of First")
case t if t =:= typeOf[Second] => println("instance of Second")
case t => println(s"unexpected type $t")
}
}
}
Example
val a = Option(First(3)) // we have a value here
val b: Option[First] = None // instance of First
val c: Option[Second] = None // instance of Second
val d: Option[Int] = None // unexpected type Int
By the way, you are interested in the type of A (which is being eliminated by erasure), so even with manifests you need one on A and not on Option[A].
Related
i am trying to make some scala functions that would help making flink map and filter operations that redirect their error to a dead letter queue.
However, i'm struggling with scala's type erasure which prevents me from making them generic. The implementation of mapWithDeadLetterQueue below does not compile.
sealed trait ProcessingResult[T]
case class ProcessingSuccess[T,U](result: U) extends ProcessingResult[T]
case class ProcessingError[T: TypeInformation](errorMessage: String, exceptionClass: String, stackTrace: String, sourceMessage: T) extends ProcessingResult[T]
object FlinkUtils {
// https://stackoverflow.com/questions/1803036/how-to-write-asinstanceofoption-in-scala
implicit class Castable(val obj: AnyRef) extends AnyVal {
def asInstanceOfOpt[T <: AnyRef : ClassTag] = {
obj match {
case t: T => Some(t)
case _ => None
}
}
}
def mapWithDeadLetterQueue[T: TypeInformation,U: TypeInformation](source: DataStream[T], func: (T => U)): (DataStream[U], DataStream[ProcessingError[T]]) = {
val mapped = source.map(x => {
val result = Try(func(x))
result match {
case Success(value) => ProcessingSuccess(value)
case Failure(exception) => ProcessingError(exception.getMessage, exception.getClass.getName, exception.getStackTrace.mkString("\n"), x)
}
} )
val mappedSuccess = mapped.flatMap((x: ProcessingResult[T]) => x.asInstanceOfOpt[ProcessingSuccess[T,U]]).map(x => x.result)
val mappedFailure = mapped.flatMap((x: ProcessingResult[T]) => x.asInstanceOfOpt[ProcessingError[T]])
(mappedSuccess, mappedFailure)
}
}
I get:
[error] FlinkUtils.scala:35:36: overloaded method value flatMap with alternatives:
[error] [R](x$1: org.apache.flink.api.common.functions.FlatMapFunction[Product with Serializable with ProcessingResult[_ <: T],R], x$2: org.apache.flink.api.common.typeinfo.TypeInformation[R])org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator[R] <and>
[error] [R](x$1: org.apache.flink.api.common.functions.FlatMapFunction[Product with Serializable with ProcessingResult[_ <: T],R])org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator[R]
[error] cannot be applied to (ProcessingResult[T] => Option[ProcessingSuccess[T,U]])
[error] val mappedSuccess = mapped.flatMap((x: ProcessingResult[T]) => x.asInstanceOfOpt[ProcessingSuccess[T,U]]).map(x => x.result)
Is there a way to make this work ?
Ok, i'm going to answer my own question. I made a couple of mistakes:
First of all, i accidentically included the java DataStream class instead of the scala DataStream class (this happens all the time). The java variant obviously doesn't accept a scala lambda for map/filter/flatmap
Second, sealed traits are not supported by flinks serialisation. There is a project that should solve it but I didn't try it yet.
Solution: first i didn't use sealed trait but a simple case class with two Options (bit less expressive, but still works):
case class ProcessingError[T](errorMessage: String, exceptionClass: String, stackTrace: String, sourceMessage: T)
case class ProcessingResult[T: TypeInformation, U: TypeInformation](result: Option[U], error: Option[ProcessingError[T]])
Then, i could have everything working like so:
object FlinkUtils {
def mapWithDeadLetterQueue[T: TypeInformation: ClassTag,U: TypeInformation: ClassTag]
(source: DataStream[T], func: (T => U)):
(DataStream[U], DataStream[ProcessingError[T]]) = {
implicit val typeInfo = TypeInformation.of(classOf[ProcessingResult[T,U]])
val mapped = source.map((x: T) => {
val result = Try(func(x))
result match {
case Success(value) => ProcessingResult[T, U](Some(value), None)
case Failure(exception) => ProcessingResult[T, U](None, Some(
ProcessingError(exception.getMessage, exception.getClass.getName,
exception.getStackTrace.mkString("\n"), x)))
}
} )
val mappedSuccess = mapped.flatMap((x: ProcessingResult[T,U]) => x.result)
val mappedFailure = mapped.flatMap((x: ProcessingResult[T,U]) => x.error)
(mappedSuccess, mappedFailure)
}
}
the flatMap and filter functions look very similar, but they use a ProcessingResult[T,List[T]] and a ProcessingResult[T,T] respectively.
I use the functions like this:
val (result, errors) = FlinkUtils.filterWithDeadLetterQueue(input, (x: MyMessage) => {
x.`type` match {
case "something" => throw new Exception("how how how")
case "something else" => false
case _ => true
}
})
import scala.reflect.runtime.universe
import scala.reflect.runtime.universe._
def getType[T: TypeTag](obj: T) = typeOf[T]
case class Thing(
val id: Int,
var name: String
)
val thing = Thing(1, "Apple")
val dataType = getType(thing).decl(TermName("id")).asTerm.typeSignature
dataType match {
case t if t =:= typeOf[Int] => println("I am Int")
case t if t =:= typeOf[String] => println("String, Do some stuff")
case _ => println("Absurd")
}
Not able to digest why result is Absurd instead of I am int?
My aim is to know data-type of class parameter at runtime and match it to predefined types.
Both current dataType and typeOf[Int] are printed as Int but if you do showRaw you'll see why they don't match
showRaw(dataType) // NullaryMethodType(TypeRef(ThisType(scala), scala.Int, List()))
showRaw(typeOf[Int]) // TypeRef(ThisType(scala), scala.Int, List())
The thing is that just the type Int and the type of nullary method returning Int are different types.
Try to add .resultType
val dataType = getType(thing).decl(TermName("id")).asTerm.typeSignature.resultType
dataType match {
case t if t =:= typeOf[Int] => println("I am Int")
case t if t =:= typeOf[String] => println("String, Do some stuff")
case _ => println("Absurd")
} // I am Int
It's also worth mentioning that .decl(TermName("id")) returns getter symbol, it's .decl(TermName("id ")) (with a blank space) that returns field symbol. So alternatively you can do with a blank space in the symbol name and without .resultType
val dataType = getType(thing).decl(TermName("id ")).asTerm.typeSignature
I'll add to #TomerShetah's answer that if the goal is "pattern matching" all fields of a case class then this can be done also at compile time (mostly) with Shapeless:
import shapeless.Poly1
import shapeless.syntax.std.product._
object printTypes extends Poly1 {
implicit val int: Case.Aux[Int, Unit] = at(t => println(s"I am Int: $t"))
implicit val string: Case.Aux[String, Unit] = at(t => println(s"String, Do some stuff: $t"))
implicit def default[V]: Case.Aux[V, Unit] = at(t => println(s"Absurd: $t"))
}
thing.toHList.map(printTypes)
// I am Int: 1
// String, Do some stuff: Apple
https://scastie.scala-lang.org/DmytroMitin/N4Idk4KcRumQJZE2CHC0yQ
#Dmytrio answer is a great explanation why the reflection didn't work as you expected.
I can understand from your question, that what you are trying to do, is actually pattern match all variables you have in a case class. Please consider doing it in the following way:
case class Thing(id: Int, name: String)
val thing = Thing(1, "Apple")
thing.productIterator.foreach {
case t: Int => println(s"I am Int: $t")
case t: String => println(s"String, Do some stuff: $t")
case t => println(s"Absurd: $t")
}
Code run at Scastie.
I have a class which contains a sequence of a generic type like:
sealed trait Interface {}
case class Imp1() extends Interface {}
case class Imp2() extends Interface {}
case class Wrapper[+I <: Interface](interface: I) {}
case class WrapperList(list: Seq[Wrapper[Interface]]) {
...
}
In the WrapperList I want to be able to iterate through the Sequence of wrappers and pattern match on each ones type, eg.
def getImp1s(remaining: Seq[Wrapper[Interface]] = list): Seq[Wrapper[Imp1]] = {
if (remaining.length == 0) Seq()
else remaining.head match {
case wrapper: Wrapper[Imp1] => get(remaining.tail) :+ wrapper
case _ => get(remaining.tail)
}
}
As you can probably guess I'm running into
non-variable type argument Playground.Imp1 in type pattern Playground.Wrapper[Playground.Imp1] is unchecked since it is eliminated by erasure
To overcome this I was under the impression I could use TypeTags or ClassTags to preserve the type, something like:
case class Wrapper[+I <: Interface](interface: I)(implicit tag: TypeTag[I]) {}
However this doesn't seem to work, I still get the same warning. Could someone explain how I can match using the TypeTag? I'd rather avoid creating concrete versions of my generic class which extend a generic abstract class, but do understand that this is maybe the easiest solution.
Thanks for your help :)
Try
import shapeless.TypeCase
val `Wrapper[Imp1]` = TypeCase[Wrapper[Imp1]]
def getImp1s(remaining: Seq[Wrapper[Interface]]): Seq[Wrapper[Imp1]] = {
if (remaining.isEmpty) Seq()
else remaining.head match {
case `Wrapper[Imp1]`(wrapper) => getImp1s(remaining.tail) :+ wrapper
case _ => getImp1s(remaining.tail)
}
}
getImp1s(Seq(Wrapper(Imp1()), Wrapper(Imp2()), Wrapper(new Interface {})))
// List(Wrapper(Imp1()))
getImp1s(Seq(Wrapper(Imp2()), Wrapper(Imp1()), Wrapper(new Interface {})))
// List(Wrapper(Imp1()))
The same can be achieved without Shapeless with custom extractor
object `Wrapper[Imp1]` {
def unapply(arg: Any): Option[Wrapper[Imp1]] = arg match {
case Wrapper(Imp1()) => Some(Wrapper(Imp1()))
case _ => None
}
}
or directly
def getImp1s(remaining: Seq[Wrapper[Interface]]): Seq[Wrapper[Imp1]] = {
if (remaining.isEmpty) Seq()
else remaining.head match {
case Wrapper(Imp1()) => getImp1s(remaining.tail) :+ Wrapper(Imp1())
case _ => getImp1s(remaining.tail)
}
}
or
def getImp1s(remaining: Seq[Wrapper[Interface]]): Seq[Wrapper[Imp1]] = {
if (remaining.isEmpty) Seq()
else remaining.head match {
case Wrapper(_: Imp1) => getImp1s(remaining.tail) :+ Wrapper(Imp1())
case _ => getImp1s(remaining.tail)
}
}
import scala.reflect.runtime.universe._
val a: 42 = 42
val t: Type = typeOf[a.type]
assert(getConstantType(t).get =:= typeOf[42])
def getConstantType(t: Type): Option[ConstantType] = ???
How could I generally implement getConstantType so that the above assertion passes?
I assumed that something like this was possible since the assertion below passes:
assert(t <:< typeOf[42])
t.widen goes too far as it return Int. I'm looking for something that returns Int(42).
How about
assert(t.resultType =:= typeOf[42])
Updated -
def getConstantType[T](t: T): t.type = t
Update 2 -
def getConstantType(tp: Type): Option[ConstantType] = {
tp.erasure match {
case ConstantType(_) => Some(tp.erasure.asInstanceOf[ConstantType])
case _ => None
}
}
Try
def getConstantType(tp: Type): Option[ConstantType] = {
def unrefine(t: Type): Type = t.dealias match {
case RefinedType(List(t), scope) if scope.isEmpty => unrefine(t)
case t => t
}
unrefine(tp) match {
case SingleType(_, sym) => sym.typeSignature match {
case NullaryMethodType(t) => unrefine(t) match {
case c: ConstantType => Some(c)
case _ => None
}
case _ => None
}
case _ => None
}
}
Here is the code
class Result[A] {
def map[B](f: (Result[A] => Result[B]), xResult: Result[A]) = {
xResult match {
case Success(x) => Success(f(xResult))
case Failure(errs) => Failure(errs)
}
}
def apply[B](fResult: Result[A], xResult: Result[B]) = {
(fResult, xResult) match {
case (Success(f), Success(x)) => Success((f, x))
case (Failure(errs), Success(a)) => Failure(errs)
case (Success(a), Failure(errs)) => Failure(errs)
case (Failure(errs), Failure(errs2)) => Failure(List(errs, errs2))
}
}
}
case class Success[A](a: A) extends Result[A] {}
case class Failure[A](a: A) extends Result[A] {}
def createCustomerId(id: Int) = {
if (id > 0)
Success(id)
else
Failure("CustomerId must be positive")
}
Here are the questions
1) result from type inference from method createCustomer is like this
Product with Serializable with Result[_ >: Int with String]
I dont have trouble with the "Product with Serializable" stuff, the thing im wondering is how to do something meaningfull with the result, and just out of curiosity how to initialize a type like that.
suppose i have another method as shown below
def createCustomer(customerId: Result[_ >: Int with String], email: Result[String]) = {
how can i read/do something with "customerId" argument
}
how to initialize a type like that
val x: Result[Int with String] = ???
notes:
i know my Result class doesnt have any generic "getter"
I know that perhaps things could be easier in method "createCustomerId" if when id > 0 i returned Success(id.ToString), but i really want to know whats going on with this lower type and how i could take advantage (if possible) in the future
The problem is that your type definition states that both Success and Failure take the same argument type. However, you actually want to put some other type of argument into Failure while still keeping covariance. Here is the simplest fix:
class Result[A] {
def map[B](f: (Result[A] => Result[B]), xResult: Result[A]) = {
xResult match {
case Success(x) => Success(f(xResult))
case Failure(errs) => Failure(errs)
}
}
def apply[B](fResult: Result[A], xResult: Result[B]) = {
(fResult, xResult) match {
case (Success(f), Success(x)) => Success((f, x))
case (Failure(errs), Success(a)) => Failure(errs)
case (Success(a), Failure(errs)) => Failure(errs)
case (Failure(errs), Failure(errs2)) => Failure(List(errs, errs2))
}
}
}
case class Success[A](a: A) extends Result[A] {}
case class Failure[A, B](a: B) extends Result[A] {}
object TestResult extends App {
def createCustomerId(id: Int): Result[Int] = {
if (id > 0)
Success(id)
else
Failure("CustomerId must be positive")
}
println(createCustomerId(0))
println(createCustomerId(1))
}
prints:
Failure(CustomerId must be positive)
Success(1)
If you don't annotate return type for createCustomerId with Result[Int] compiler will infer Result[_ <: Int] with Product with Serializable which is probably not ideal but not too bad.