Generic method arguments in scala - scala

I'm not sure the title is describing my question the best but let's give it a shot.
I have a background job execution application that resembles a simple pipeline processing. There are Command objects that do some calculation and return an OUTPUT, and Worker that receive OUTPUT as input and can return Result
The object model looks something like this:
type OUTPUT <: AnyRef
trait Command[OUTPUT] {
def doSomething(): OUTPUT
}
sealed trait Worker[IN <: AnyRef, OUT <: Result] {
def work(input: IN): OUT
}
case class WorkA() extends Worker[String, LongResult] {
override def work(input: String): LongResult = LongResult(Long.MaxValue)
}
case class WorkB() extends Worker[Long, StringResult] {
override def work(input: Long): StringResult = StringResult(input.toString)
}
There are few problems with this approach:
When mapping on a collection of Worker I can't make sure the worker accepts the same OUTPUT as input.
Unless I'm mapping a List of Command, the code does not compile because of type erasure - it expects a _$1 but receives a Long or a String (everything that was previously accepted as OUTPUT)
val workers = List(
new WorkA(),
new WorkB()
)
val aSimpleCommand = new Command[Long] {
override def doSomething() = 123123123L
}
// Obviously this doesn't compile.
workers.map(worker => worker.work(aSimpleCommand.doSomething()))
I'm looking for the right Scala mechanism to disallow this at compile time. How can I map ONLY on the Worker that DO actually support OUTPUT - and in this case, only WorkB

If you want to do this at compile time you can use shapeless HLists, preserving the type of your list all the way through, and then using a Poly to handle the cases:
val myWorkers: WorkA :: WorkB :: WorkB :: HNil =
WorkA() :: WorkB() :: WorkB() :: HNil
object doWork extends Poly1 {
implicit def caseLong[A] = at[Worker[Long, A]] {
w => w.work(aSimpleCommand.doSomething())}
implicit def caseString = //whatever you want to do with a String worker
}
myWorkers map doWork
For a less safe example that doesn't need shapeless, you can match cases as long as you have concrete types:
val myWorkers: List[Worker[_, _]] = ...
myWorkers collect {
case wa: WorkA => //works
case lw: Worker[Long, _] => //doesn't work
}

If it is feasible to explicitly state all classes that extend Worker[Long, _], then you could map the workers over a partial function like so:
val res = workers map {
case b: WorkB => Some(b.work(aSimpleCommand.doSomething()))
case _ => None
}
In your example, this will return List(None, Some(StringResult(123123123)). You can also collect only the existing values:
res collect {case Some(r) => r} // List(StringResult(123123123))
Now, this isn't a very practical solution. Maybe the following thoughts help you come up with something better:
As you have already stated, because of type erasure, we cannot create our partial function to accept values of type Worker[Long, _] at runtime. ClassTag (and TypeTag) provide a solution to this problem by making the compiler create evidence accessible at runtime for the erased types. For example, the following function extracts the runtime class of a worker's input:
import scala.reflect.ClassTag
def getRuntimeClassOfInput[T: ClassTag](worker: Worker[T, _]) = implicitly[ClassTag[T]].runtimeClass
Example usage:
println(getRuntimeClassOfInput(new WorkB)) // long
The problem is that this doesn't seem to work once the workers are inside a list. I suppose that it's because once you have multiple different workers in the list, the list becomes a List[Worker[Any, Result] and you lose all type information. What might solve this problem are Heterogenous Lists because unlike the standard lists they keep the static type information of all elements.

Related

How to implement subtype resolution of typeclass in scala

I want to understand how to go about implementing the following use-case using typeclasses in Scala (or find out if it is even possible).
Given a sealed trait and a couple of concrete cases:
sealed trait Base
case class Impl1() extends Base
case class Impl2() extends Base
Given a typeclass operating on Base and an instance for each of the corresponding base implementations:
trait Processor[B <: Base]:
def process(b: B): String
given Processor[Impl1] with:
def process(b: Impl1): String = ??? // not important
given Processor[Impl2] with:
def process(b: Impl2): String = ??? // not important
Given a list of base objects:
val objects: List[Base] = ??? // whatever
Is it possible to implement a method that goes something like this?
val processed = objects.map(obj => process(obj))
def process[B <: Base](b: B)(using proc: Processor[B]) = proc.process(b)
When I try to naively implement the above as-such, the compiler complains that it can't find an implicit for Processor[Base], which I guess it makes sense, since in the context of the method call for process(obj), the obj val has the Base type.
What I would like to do is to let the compiler figure out the concrete type of obj, fetch the corresponding given instance for the concrete type and inject it into the process method. Is it even possible to do such a thing? Does it even make sense?
(Note - I've written my code in scala 3, but I'll gladly accept an answer in scala 2 syntax).
With a list of base objects you won't be able to do it, the list doesn't contain the real type at compile time and the compiler won't be able to find their respective type classes.
In scala 3 the typed list is the tuple, so you can use a tuple to have a typed list and obtain what you are expecting. The method you need is and adaptation of the one you can find with the same name in the scala 3 documentation and is simple as:
inline def summonAll[T <: Tuple](tup: T): List[String] =
inline tup match
case _: EmptyTuple => Nil
case tupl: (t *: ts) => summonInline[Processor[t]].process(tupl.head) :: summonAll[ts](tupl.tail)
and you can use it only passing the list of elements you want to obtain in a tuple in the expected type:
summonAll[(A, B)]
Here you can see the following full code running:
import scala.compiletime.{erasedValue, summonInline}
trait Processor[A]:
def process(t: A): String
object Processor:
def apply[T: Processor]: Processor[T] = summon[Processor[T]]
given Processor[String] with
def process(t: String): String = "im String: " + t
given Processor[Int] with
def process(t: Int): String = "im Int: " + t
inline def summonAll[T <: Tuple](tup: T): List[String] =
inline tup match
case _: EmptyTuple => Nil
case tupl: (t *: ts) => summonInline[Processor[t]].process(tupl.head) :: summonAll[ts](tupl.tail)
val t1 = ("hi", "bye", 44 )
val t2 = summonAll(t1)
println(t2) //List(im String: hi, im String: bye, im Int: 44)
With this particular typeclass you can easily create the Processor[Base] which the compiler asks for:
given Processor[Base] with:
def process(b: Base): String = b match
case b: Impl1 => process(b)
case b: Impl2 => process(b)
Using type class derivation you can also generate it automatically, but I am afraid it's going to be far more code! And if you enable warnings (which you should), the compiler should warn you about a non-exhaustive match if a new subclass is added anyway. So I'd use this manual version unless you have quite a lot of subclasses to handle in your actual case or they change often.
Note "this particular typeclass" it wouldn't work e.g. if process took more than 1 B parameter, or some parameters with more complex types containing B.

List of classes implementing a certain typeclass

I would like to define a List of elements implementing a common type class. E.g.
trait Show[A] {
def show(a: A): String
}
implicit val intCanShow: Show[Int] = new Show[Int] {
def show(int: Int): String = s"int $int"
}
implicit val stringCanShow: Show[String] = new Show[String] {
def show(str: String): String = str
}
The problem is, how to define a list = List(1, "abc") such that it is guaranteed that a Show instance for these values is in scope? I would then like to map this list over show like list map {_.show}.
I will first sketch a solution, and then explain why the naive approach with List[Any](1, "abc") cannot work.
What you can do
Define a wrapper class that can hold instances of type A together with instances of Show[A]:
case class Showable[A](a: A, showInst: Show[A]) {
def show: String = showInst.show(a)
}
Define your list as List[Showable[_]]:
var showableList: List[Showable[_]] = Nil
Maybe define a separate method to fill this list (consider packing the list itself and the builder-method in a class):
def addShowable[A: Show](a: A): Unit = {
showableList ::= Showable[A](a, implicitly[Show[A]])
}
Alternatively, you can carefully add a (very tightly scoped) implicit conversion:
implicit def asShowable[A](a: A)(implicit s: Show[A]): Showable[A] =
Showable(a, s)
and then costruct your list as follows (note the explicit type ascription):
val showableList = List[Showable[_]](1, "abc")
Now you can go through the list and call show:
showableList.map(_.show)
to obtain a list of String.
What you cannot do
You cannot simply define
val list: List[Any] = List(1, "abc", <showable3>, ..., <showableN>)
and then expect to be able to call show, because in order to call Show.show, you need actual Show instances. These things are not some type-hints that can be erased at runtime, they are actual objects, and they must be supplied by the compiler. Once you have created a List[Any], all is lost, because all the types are merged into an unexpressive upper bound Any, and the compiler has no way to inject all the necessary implicits Show[T_1],..., Show[T_N]. The argument is very similar to the third section "Dealing with implicits when defining interpreter for the Free monad" of this lengthy answer of mine.
An alternative way of handling this would be to use the shapeless library.
I would really reccommend this book which explains shapeless in a clear and concise manner.
Shapeless provides two things that I think will help you in this case:
Heterogeneous lists (HList)
Polymorphic functions to enable the HList mapping operation.
First import the required libraries (shapeless):
import shapeless.{HNil, Poly1, ::}
Create a heterogeneous list of whatever types you require. Note the type annotation is only there for clarity.
val data : Int :: String :: HNil = 1 :: "hello" :: HNil
Create a polymorphic function defining an implicit value for every type you require.
object Show extends Poly1 {
implicit def atT[T: Show] = at[T] (implicitly[Show[T]].show)
}
Shapeless provides an extension method for map on a HList to enable applying the show function to every element in the list
val result : String :: String :: HNil = data.map(Show)
Edited: thanks to #dk14 for the suggested improvement to the definition of the Show polymorphic function.
The core problem here is that you want to create a heterogenous list, something like List[Int, String] instead of List[Any]. This means you need a different structure that would preserve Int and String types, but still would be "mappable" like List. The one structure in scala-library that can contain heterogenous types is Tuple:
val tuple = (1, "abc")
val result = List(implicitly[Show[Int]].show(tuple._1), implicitly[Show[Int]].show(tuple._2))
However, scala-library can't map over tuples - you might want some syntax sugar for better readability.
So the obvious solution is HList from Shapeless: Int :: String :: HNil (or you can use tuple ops and stay with (Int, String))
import shapeless._
import poly._
//show is a polymorphic function
//think of it as `T => String` or even `(Show[T], T) => String`
object show extends Poly1 {
implicit def atT[T: Show] = at[T](implicitly[Show[T]].show)
}
# (1 :: "aaaa" :: HNil) map show
res8: String :: String :: HNil = "int 1" :: "aaaa" :: HNil
Or you could use at[Int]/at[String] instead of type-classes, like in #Steve Robinson's answer.
P.S. The lib could be found here. They also provide one-liner to get Ammonite REPL with shapeless integrated, so you could try my example out using:
curl -s https://raw.githubusercontent.com/milessabin/shapeless/master/scripts/try-shapeless.sh | bash
Notes:
Practically Shapeless solution requires as same amount of maintenance as Tuple-based one. This is because you have to keep track of your Int and String types anyways - you can never forget about those (unlike in homogenous List[T] case). All Shapeless does for you is nicer syntax and sometimes better type inference.
If you go with tuples - you can improve readability by using implicit class instead of Haskell-like style, or if you still want Haskell-like, there is a Simulacrum macro for better type-class syntax.
Given that other scala-library-only alternatives just capture type class instances inside some regular class, you could be better off with a regular OOP wrapper class:
trait Showable[T]{def value: T; def show: String}
class IntShow(val value: Int) extends Showable[Int]{..}
class StringShow(val value: String) extends Showable[String] {..}
val showables: List[Showable[_]] = List(new Showable(5), new Showable("aaa"))
showables.map(_.show)
Looks cleaner and more readable to me :)
If you like to rewrite dynamic dispatching in FP-style:
sealed trait Showable
final case class ShowableInt(i: Int) extends Showable
final case class ShowableString(s: String) extends Showable
implicit class ShowableDispatch(s: Showable){
def show = s match{ //use `-Xfatal-warnings` scalac option or http://www.wartremover.org/ to guarantee totality of this function
case ShowableInt(i) => ...
case ShowableString(s) => ...
}
}
List(ShowableInt(5), ShowableString("aaa")).map(_.show)
If you really want static dispatching (or ad-hoc polymorphism), given that other solutions introduce Showable[_] which is practically Showable[Any]:
case class Showable[T](v: T, show: String)
def showable(i: Int) = Showable(i, s"int $i")
def showable(s: String) = Showable(i, s)
List(showable(5), showable("aaa")).map(_.show)
Disclamer: I've made this answer to provide a solution to the concrete development problem, and not the theoretical problem of using typeclass
I would do it this way:
trait Showable{ def show(): String }
implicit class IntCanShow(int: Int) extends Showable {
def show(): String = s"int $int"
}
implicit class StringCanShow(str: String) extends Showable {
def show(): String = str
}
val l: List[Showable] = List(1,"asd")
Note that I changed the meaning of the trait Show, into Showable, such that the implementing classes are used as wrapper. We can thus simply require that we want Showable instances
(and because those classes are implicit, input of the List are automatically wrapped)

Scala polymorphism - covariant and type bound

Using Scala... I can't figure out how to use polymorphism in a way that mixes type bound and covariance.
In a nutshell, I think I need something like this type signature... but if you follow along with my dummy example, you'll see why I get here... and maybe I'm wrong.
def func[+T <: U](func: Seq[T] => T)(iter: Iterator[String]): Map[String, String] = ???
but this approach yields...
>> error: ']' expected but identifier found
Here's a dummy example that demonstrates what I'm trying to do... I could sidestep this by just making work only with the base trait Record... but I'd like to get it working with polymorphism baked in for other reasons in the real code.
setup
// underlying trait to hold key and value
trait Record {
def k: String
def v: String
def isDefined: Boolean
}
// companion object with apply method
object Record {
def apply(s: String): Record = s.split(",") match {
case Array(k,v) => new ValidRecord(k,v).asInstanceOf[Record]
case _ => EmptyRecord.asInstanceOf[Record]
}
}
// singleton for empty records
object EmptyRecord extends Record {
val k = ""
val v = ""
val isDefined = false
}
// class for actual data
class ValidRecord(val k: String, val v: String) extends Record {
val isDefined = true
}
polymorphic function
note - going from Iterator to Seq here looks questionable... I'm reading in a file from src/main/resources... it comes in as an Iterator... and I ultimately need to get it into a Map, so .toSeq and .groupBy seem like logical steps... it's only maybe 100MB and a million or so records, so this works fine... but if there's a smarter way to get from start to end, I'm open to that critique as well.
def iter_2_map[T <: Record](func: Seq[T] => T)(iter: Iterator[String]): Map[String, String] = {
iter // iterator of raw data
.map(Record.apply) // Iterator[Record]
.toSeq // gives .groupBy() method
.groupBy(_.k) // Map[k -> Seq[Record]]; one Seq of records per k
.mapValues(func) // <<< ERROR HERE //function to reduce Seq[Record] to 1 Record
.filter(_._2.isDefined) // get rid of empty results
.mapValues(_.v) // target of Map is just v
}
error
found : Seq[T] => T
required: Seq[Record] => ?
.mapValues(func)
^
If I break down all those steps and declare types at every relevant step... the error changes to this...
found : Seq[T] => T
required: Seq[Record] => Record
.mapValues(func)
^
So here's where I get stumped. I think making T covariant solves this... T is a declared subtype of Record, but maybe it's not recognizing Seq[T] as <: Seq[Record]?
But making this change yields the error at the top...
def iter_2_map[+T <% Record](func: Seq[T] => T)(iter: Iterator[String]): Map[String, String] = {
???
}
back to this...
>> error: ']' expected but identifier found
Am I even on the right track?
You are using + incorrectly. It's only used with type parameters of classes to signal that the class should be covariant in its parameter.
It does not make very much sense to use it with methods (Seq[T] actually is a subclass of Seq[Record] - because Seq is covariant, but that does not help you, because functions are contravariant in their argument type, so Function[Seq[T], T] is a superclass of Function[Seq[Record], T], not a subclass). Here is why:
After .groupBy(_.k) you have Map[String, Seq[Record]].
Now, you are doing .mapValues(func) on it, and are trying to pass a function to it, that takes a Seq[T]. This cannot work.
Imagine, that Record is Animal, and T is Dog ... and func is makeBark ... And now you are trying to pass a bunch of animals to it, some of which are Cats, some Birds, and some, maybe Fish. You can't make them all bark, can you?
You could just declare your reducer function to accept the Record sequence rather than T:
def iter_2_map[T <: Record](func: Seq[Record] => T)(iter: Iterator[String])
This will compile, but doesn't seem like it would be very useful for you anyway, because you appear to expect your func to be able to return both EmptyRecord and ValidRecord, and not just T (since you are filtering the empties out afterwards). So, it actually seems that you don't need the type parameter at all after all:
def iter_2_map(func: Seq[Record] => Record)(iter: Iterator[String])

Diverging implicit expansion for type class

I'm trying to extend the Parser type class from this blog post to support nested case classes parsing. Since I'm new to shapeless, I've decided to start with the non-general case, that would only support a case class with 2 fields that are themselves case classes. To illustrate this better, here's the case classes definition first:
case class Person(name: String, age: Int)
case class Family(husband: Person, wife: Person)
In addition to the code from that blog post, I've created one more implicit function that is supposed to support my case, parsing Family class from the string like "Hans,70|Emmy,60" by dividing the string in 2 parts by '|', then parsing those 2 separate parts independently, and at last combining the result. Here how it looks like:
implicit def nestedCaseClassParser[A, B, C]
(
implicit pb: Parser[B],
pc: Parser[C],
gen: Generic.Aux[A, B :: C :: HNil]
): Parser[A] = new Parser[A] {
override def apply(s: String): Option[A] = {
val tmp = s.span(_ != '|') match {
case (h, t) =>
for {
a <- pb(h)
b <- pc(t.substring(1))
} yield a :: b :: HNil
}
tmp.map(gen.from)
}
}
Then, when I'm trying to parse it like this:
val p = Parser.apply[Family]("hello,1|hello,1")
I'm getting the following compilation error:
<console>:83: error: diverging implicit expansion for type Parser[Family]
starting with method nestedCaseClassParser in object Parser
val p = Parser.apply[Family]("hello,1|hello,1")
^
I'm not asking for a full solution for the problem of fully generic nested case class parsing, as I'm eager to find it myself, but I'd really appreciate an explanation of what's going wrong with this code, and how can I make it work on this simple level.
EDIT: Since I've been able to overcome the initial problem, I've created a separate questions for the thing I've run into next.

Evidence-preserving LUB constraint for HList

I think I need a HList that is constrained to have all of its elements being a subtype of a certain type. LUBConstraint seems to be what I want, and indeed it does constrain the construction of such a HList - but I can't see how to get the evidence out again, so that I can map (actually, traverse, because it needs to be monadic) over the HList and call a method (that exists in the LUB type) on each of the elements.
In addition, I want the type of the HList resulting from the traverse operation to be exactly the same type as the type of the input HList.
The use case is a kind of functional "listener list" - all of the elements of the HList are "listeners" which must be notified of "events", accept or reject them, and return new versions of themselves with updated "internal state". If that was all I needed, then I could just use an ordinary immutable Scala collection. But I also want direct typed access to individual elements without using asInstanceOf - hence the motivation for trying to use HList.
In general if you have some operation that you want to perform on all of the elements in an HList, you'll want to map a polymorphic function value over the HList. For example, suppose I have the following setup:
trait Listener[L <: Listener[L]] {
def handle(s: String): Option[L]
}
class FooListener extends Listener[FooListener] {
def handle(s: String) =
if (s.size == 3) Some(this) else None
}
class BarListener extends Listener[BarListener ]{
def handle(s: String) = Some(this)
}
import shapeless._
val listeners = new FooListener :: new BarListener :: HNil
Now I want to send a String to each of these listeners and gather the results. If I just wanted to send a fixed value, this would be easy:
object event123 extends Poly1 {
implicit def listener[L <: Listener[L]] = at[L](_.handle("123"))
}
val result = listeners.map(event123)
Which will be appropriately typed as an Option[FooListener] :: Option[BarListener] :: HNil. If I'm using shapeless-contrib, I can sequence this HList:
import scalaz._, Scalaz._, shapeless.contrib.scalaz._
val sequenced: Option[FooListener :: BarListener :: HNil] = sequence(result)
Or just use traverse:
traverse(listeners)(event123)
Unfortunately there are restrictions on how polymorphic function values can be defined that mean that partial application isn't convenient, so if we don't know the String we're sending at compile time, this is a lot more complicated:
object event extends Poly1 {
implicit def listener[L <: Listener[L]] = at[(L, String)] {
case (listener, string) => listener.handle(string)
}
}
traverse(listeners.zip(listeners.mapConst("123")))(event)
Where we've zipped the elements with the string and then mapped a polymorphic function that takes tuples over the result. There are other ways you could do this using more or less the same approach, but none of them are terribly clear.
A completely different approach is just to skip the polymorphic function values and define a new type class:
trait Notifiable[L <: HList] {
def tell(s: String)(l: L): Option[L]
}
object Notifiable {
implicit val hnilNotifiable: Notifiable[HNil] = new Notifiable[HNil] {
def tell(s: String)(l: HNil) = Some(HNil)
}
implicit def hconsNotifiable[H <: Listener[H], T <: HList](implicit
tn: Notifiable[T]
): Notifiable[H :: T] = new Notifiable[H :: T] {
def tell(s: String)(l: H :: T) = for {
h <- l.head.handle(s)
t <- tn.tell(s)(l.tail)
} yield h :: t
}
}
def tell[L <: HList: Notifiable](s: String)(l: L) =
implicitly[Notifiable[L]].tell(s)(l)
And then:
val sequenced: Option[FooListener :: BarListener :: HNil] =
tell("123")(listeners)
This is less generic (it only works on Option, not arbitrary applicatives), but it doesn't require an extra dependency for sequencing, and it's arguably a little less muddled than jumping through hoops to partially apply a polymorphic function value because of weird limitations of the compiler.