This is a followup to my previous question with an example found on the Internet.
Suppose I define a typeclass Applicative as follows:
trait Functor[T[_]]{
def map[A,B](f:A=>B, ta:T[A]):T[B]
}
trait Applicative[T[_]] extends Functor[T] {
def unit[A](a:A):T[A]
def ap[A,B](tf:T[A=>B], ta:T[A]):T[B]
}
I can define an instance of Applicative for List
object AppList extends Applicative[List] {
def map[A,B](f:A=>B, as:List[A]) = as.map(f)
def unit[A](a: A) = List(a)
def ap[A,B](fs:List[A=>B], as:List[A]) = for(f <- fs; a <- as) yield f(a)
}
For convenience I can define an implicit conversion to add a method <*> to List[A=>B]
implicit def toApplicative[A, B](fs: List[A=>B]) = new {
def <*>(as: List[A]) = AppList.ap(fs, as)
}
Now I can do a cool thing !
zip two lists List[String] and apply f2 to every pair in applicative style
val f2: (String, String) => String = {(first, last) => s"$first $last"}
val firsts = List("a", "b", "c")
val lasts = List("x", "y", "z")
scala> AppList.unit(f2.curried) <*> firsts <*> lasts
res31: List[String] = List(a x, a y, a z, b x, b y, b z, c x, c y, c z)
So far, so good but now I have:
val firstsOpt = Some(firsts)
val lastsOpt = Some(lasts)
I would like to zip firsts and lasts, apply f2, and get Option[List[String]] in applicative style. In other words I need <*> for Option[List[_]]. How can I do it ?
Firstly, you need an instance of applicative for Option:
implicit object AppOption extends Applicative[Option] {
def map[A, B](f: A => B, o: Option[A]) = o.map(f)
def unit[A](a: A): Option[A] = Some(a)
def ap[A, B](of: Option[A => B], oa: Option[A]) = of match {
case Some(f) => oa.map(f)
case None => None
}
}
Then you can also create an applicative instance for the composition of two applicatives (note, based on the Haskell version):
class AppComp[F[_], G[_]](fa: Applicative[F], ga: Applicative[G]) extends Applicative[({ type f[A] = F[G[A]]})#f] {
def map[A, B](f: A => B, a: F[G[A]]): F[G[B]] = fa.map((g: G[A]) => ga.map(f, g), a)
def unit[A](a: A) = fa.unit(ga.unit(a))
def ap[A, B](f: F[G[A => B]], a: F[G[A]]): F[G[B]] = {
val liftg: G[A => B] => (G[A] => G[B]) = gf => (gx => ga.ap(gf, gx))
val ffg: F[G[A] => G[B]] = fa.map(liftg, f)
fa.ap(ffg, a)
}
}
implicit def toComp[F[_], G[_]](implicit fa: Applicative[F], ga: Applicative[G]) = new AppComp[F, G](fa, ga)
Finally you can now do:
val ola = toComp[Option, List]
ola.ap(ola.ap(ola.unit(f2.curried), firstsOpt), lastsOpt)
You could probably also remove some of the noise by generalising <*> to work for any applicative.
Related
I was playing with cats' Monoids in scala when I see that the monoid operations are extended for Tuples in a natural way:
import cats.Monoid
object mon {
implicit object IntMonoid extends Monoid[Int] {
def combine(a: Int, b: Int) = a*a + b*b
def empty = 0
}
implicit object ListMonoid extends Monoid[List[Int]] {
def combine(a: List[Int], b: List[Int]): List[Int] =
a.zip(b).map(z => z._1 * z._2)
def empty = List(1)
}
def comb[T](a: T, b: T)(implicit m: Monoid[T]) =
m.combine(a, b)
}
val list1 = List(1, 2, 3)
val list2 = List(2, 3, 4)
println(mon.comb(list1, list2)) // outputs: List(2, 6, 12) as expected
val int1 = 2
val int2 = 4
println(mon.comb(int1, int2)) // outputs: 20 as expected
val x = (list1, int1)
val y = (list2, int2)
println(mon.comb(x, y)) // outputs: (List(2, 6, 12),20)
The last output is expected in a 'natural' way, but how does de compiler knows how to do it?
I've been trying to look for it in Cats' source code, but I'm not as experienced in Scala as to be able to know what to look for. I suppose the same methods holds for similar constructions like semigroups.
Your question boils down to how implicit derivation of typeclasses for generic types work; so let's see two examples:
A case where we want to provide an instance no matter what the generic is:
// Similar to the code you had, but without being tied to just List[Int],
// Since in this case the Int part is irrelevant.
implicit def monoidList[A]: Monoid[List[A]] =
new Monoid[List[A]] {
override final val empty: List[A] = Nil
override final def combine(l1: List[A], l2: List[A]): List[A] =
l1 ::: l2
}
A case where we require a proof of the generic type to provide the instance of the complex type:
implicit def optionMonoid[A](implicit aMonoid: Monoid[A]): Monoid[Option[A]] =
new Monoid[Option[A]] {
override final val empty: Option[A] = None
override final def combine(o1: Option[A], o2: Option[A]): Option[A] =
(o1, o2) match {
case (None, None) => None
case (Some(a), None) => Some(a)
case (None, Some(a)) => Some(a)
case (Some(a1), Some(a1)) => Some(aMonoid.combine(a1, a2))
}
}
Thus, you can now imagine how the Monoid[Tuple2[A, B]] of cats works, but just for completeness the code would be like this:
implicit def tuple2Monoid[A, B](implicit aMonoid: Monoid[A], bMonoid: Monoid[B]): Monoid[(A, B)] =
new Monoid[(A, B)] {
override final def empty: (A, B) =
(aMonoid.empty, bMonoid.empty)
override final def combine(t1: (A, B), t2: (A, B)): (A, B) =
(t1, t2) match {
case ((a1, b1), (a2, b2)) => (aMonoid.combine(a1, a2), bMonoid.combine(b1, b2))
}
}
I have multiple map functions running over the same data and I'd like to have them run in a single pass. I'm looking for a generic way to do this.
val fruits: Seq[String] = Seq("apple", "banana", "cherry")
def mapF(s: String): Char = s.head
def reduceF(c1: Char, c2: Char): Char = if(c1 > c2) c1 else c2
def mapG(s: String): Int = s.length
def reduceG(i1: Int, i2: Int): Int = i1 + i2
val largestStartingChar = fruits.map(mapF).reduce(reduceF)
val totalStringLength = fruits.map(mapG).reduce(reduceG)
I'd like to reduce the number of passes over fruits. I can make this generic for two maps and reduces like this:
def productMapFunction[A, B, C](f: A=>B, g: A=>C): A => (B, C) = {
x => (f(x), g(x))
}
def productReduceFunction[T, U](f: (T, T)=>T, g: (U, U) => U):
((T,U), (T,U)) => (T, U) = {
(tu1, tu2) => (f(tu1._1, tu2._1), g(tu1._2, tu2._2))
}
val xMapFG = productMapFunction(mapF, mapG)
val xReduceFG = productReduceFunction(reduceF, reduceG)
val (largestStartingChar2, totalStringLength2) =
fruits.map(xMapFG).reduce(xReduceFG))
I'd like to do this even more generically, with an arbitrary number of map and reduce functions, but I'm not sure how to proceed, or if this is possible.
Following solution uses Cats 2 and custom type MapReduce.
Reducing operation can be specified by function reduce: (O, O) => O
or cats reducer: Semigroup[O].
Multiple MapReduce objects can be combined into one by Apply instance provided by implicit def mapReduceApply[I]
import cats._
import cats.implicits._
trait MapReduce[I, O] {
type R
def reducer: Semigroup[R]
def map: I => R
def mapResult: R => O
def apply(input: Seq[I]): O = mapResult(input.map(map).reduce(reducer.combine))
}
object MapReduce {
def apply[I, O, _R](_reducer: Semigroup[_R], _map: I => _R, _mapResult: _R => O): MapReduce[I, O] =
new MapReduce[I, O] {
override type R = _R
override def reducer = _reducer
override def map = _map
override def mapResult = _mapResult
}
def apply[I, O](map: I => O)(implicit r: Semigroup[O]): MapReduce[I, O] =
MapReduce[I, O, O](r, map, identity)
def apply[I, O](map: I => O, reduce: (O, O) => O): MapReduce[I, O] = {
val reducer = new Semigroup[O] {
override def combine(x: O, y: O): O = reduce(x, y)
}
MapReduce(map)(reducer)
}
implicit def mapReduceApply[I] =
new Apply[({type F[X] = MapReduce[I, X]})#F] {
override def map[A, B](f: MapReduce[I, A])(fn: A => B): MapReduce[I, B] =
MapReduce(f.reducer, f.map, f.mapResult.andThen(fn))
override def ap[A, B](ff: MapReduce[I, (A) => B])(fa: MapReduce[I, A]): MapReduce[I, B] =
MapReduce(ff.reducer product fa.reducer,
i => (ff.map(i), fa.map(i)),
(t: (ff.R, fa.R)) => ff.mapResult(t._1)(fa.mapResult(t._2))
)
}
}
object MultiMapReduce extends App {
val fruits: Seq[String] = Seq("apple", "banana", "cherry")
def mapF(s: String): Char = s.head
def reduceF(c1: Char, c2: Char): Char = if (c1 > c2) c1 else c2
val biggestFirsChar = MapReduce(mapF, reduceF)
val totalChars = MapReduce[String, Int](_.length) // (Semigroup[Int]) reduce by _ + _
def count[A] = MapReduce[A, Int](_ => 1)
val multiMapReduce = (biggestFirsChar, totalChars, count[String]).mapN((_, _, _))
println(multiMapReduce(fruits))
val sum = MapReduce[Double, Double](identity)
val average = (sum, count[Double]).mapN(_ / _)
println(sum(List(1, 2, 3, 4)))
println(average(List(1, 2, 3, 4)))
}
Runnable version is also available on GitHub.
Interesting question!
I don't know of any such implementation in the standard library or even scalaz/cats.
It's not very surprising because if your list is not very large you can just perform map-reduces sequentially and I'm not even sure that overhead of constructing lots of intermediate objects would be smaller than overhead of traversing the list several times.
And if the list is potentially doesn't fit into memory you should be using one of the streaming libraries (fs2/zio-streams/akka-streams)
Although if your input was Iterator instead of List, such functionality would be useful.
There is an interesting article about this problem:
https://softwaremill.com/beautiful-folds-in-scala/
tldr:
Map-reduce workflow could be formalized as follows:
trait Fold[I, O] {
type M
def m: Monoid[M]
def tally: I => M
def summarize: M => O
}
In your case I = List[A], tally = list => list.map(mapF), summarize = list => list.reduce(reduceF) .
To run a map-reduce on a list using an instance of fold you need to run
fold.summarize(fold.tally(list))
You can define combine operation on them:
def combine[I, O1, O2](f1: Fold[I, O1], f2: Fold[I, O2]): Fold[I, (O1, O2)]
Using combine few times would give you what you want:
combine(combine(f1, f2), f3): Fold[I, ((O1, O2), O3)]
I think you're just trying to reinvent transducers. It's been a while since I have used Scala, but there's at least one implementation.
Note - the operation described below now exists in the standard library as partitionMap but I believe it's still a valid question as to how to achieve more general ends
Question regarding scala 2.13 - how do I consume/construct collections of specific types when adding custom collections operations where I need to restrict the element types of the input collections? e.g. how do I define:
def split[CC[_], A, B](coll: CC[Either[A, B]]): (CC[A], CC[B])
Following the documentation I've managed to achieve this as follows:
import collection.generic.IsIterable
import scala.collection.{BuildFrom, Factory}
class SplitOperation[Repr, S <: IsIterable[Repr]](coll: Repr, itr: S) {
def split[A, B, AS, BS](
implicit bfa: BuildFrom[Repr, A, AS],
bfb: BuildFrom[Repr, B, BS],
ev: itr.A =:= Either[A, B]): (AS, BS) = {
val ops = itr(coll)
val as = bfa.fromSpecific(coll)(ops.iterator.map(ev).collect { case Left(a) => a })
val bs = bfb.fromSpecific(coll)(ops.iterator.map(ev).collect { case Right(b) => b })
(as, bs)
}
}
implicit def SplitOperation[Repr](coll: Repr)(implicit itr: IsIterable[Repr]): SplitOperation[Repr, itr.type] =
new SplitOperation(coll, itr)
However, I need to supply types at the use-site otherwise I get diverging implicit expansion.
scala> List(Left("bah"), Right(1), Left("gah"), Right(2), Right(3))
res1: List[scala.util.Either[String,Int]] = List(Left(bah), Right(1), Left(gah), Right(2), Right(3))
scala> res1.split
^
error: diverging implicit expansion for type scala.collection.BuildFrom[List[scala.util.Either[String,Int]],A,AS]
But the following works:
scala> res1.split[String, Int, List[String], List[Int]]
res4: (List[String], List[Int]) = (List(bah, gah),List(1, 2, 3))
EDIT
class SplitOperation[X, CC[_], S <: IsIterable[CC[X]]](coll: CC[X], itr: S) {
def split[A, B](implicit bfa: BuildFrom[CC[X], A, CC[A]], bfb: BuildFrom[CC[X], B, CC[B]], ev: itr.A =:= Either[A, B]): (CC[A], CC[B]) = {
val ops = itr(coll)
val as = bfa.fromSpecific(coll)(ops.iterator.map(ev).collect { case Left(a) => a })
val bs = bfb.fromSpecific(coll)(ops.iterator.map(ev).collect { case Right(b) => b })
(as, bs)
}
}
implicit def SplitOperation[A, B, CC[_]](coll: CC[Either[A, B]])(implicit itr: IsIterable[CC[Either[A, B]]]): SplitOperation[Either[A, B], CC, itr.type] =
new SplitOperation(coll, itr)
Gives me a slight improvement. Now I only need to provide type parameters A and B at the call site:
scala> l.split[String, Int]
res2: (List[String], List[Int]) = (List(bah, gah),List(1, 2))
This seems to work:
class SplitOperation[A, B, CC[_], S <: IsIterable[CC[Either[A, B]]]](coll: CC[Either[A, B]], itr: S) {
def split(implicit bfa: BuildFrom[CC[Either[A, B]], A, CC[A]], bfb: BuildFrom[CC[Either[A, B]], B, CC[B]], ev: itr.A =:= Either[A, B]): (CC[A], CC[B]) = {
val ops = itr(coll)
val as = bfa.fromSpecific(coll)(ops.iterator.map(ev).collect { case Left(a) => a })
val bs = bfb.fromSpecific(coll)(ops.iterator.map(ev).collect { case Right(b) => b })
(as, bs)
}
}
implicit def SplitOperation[A, B, CC[_]](coll: CC[Either[A, B]])(implicit itr: IsIterable[CC[Either[A, B]]]): SplitOperation[A, B, CC, itr.type] =
new SplitOperation(coll, itr)
In your case you don’t want to abstract over the “kind” of the collection type constructor (CC[_] vs CC[_, _], etc.), you always use the CC[_] kind, so you don’t need to use IsIterable.
I think it is also not necessary to support “Sorted” collections (eg, SortedSet) because there is no Ordering instance for Either, so you don’t need to use BuildFrom.
implicit class SplitOperation[A, B, CC[X] <: IterableOps[X, CC, CC[X]]](coll: CC[Either[A, B]]) {
def split: (CC[A], CC[B]) = {
val as = coll.iterableFactory.from(coll.iterator.collect { case Left(a) => a })
val bs = coll.iterableFactory.from(coll.iterator.collect { case Right(b) => b })
(as, bs)
}
}
https://scastie.scala-lang.org/64QxHwteQN2i3udSxCa3yw
What is the best way to partition Seq[A \/ B] into (Seq[A], Seq[B]) using Scalaz?
There is a method: separate defined in MonadPlus. This typeclass is a combination a Monad with PlusEmpty (generalized Monoid). So you need to define instance for Seq:
1) MonadPlus[Seq]
implicit val seqmp = new MonadPlus[Seq] {
def plus[A](a: Seq[A], b: => Seq[A]): Seq[A] = a ++ b
def empty[A]: Seq[A] = Seq.empty[A]
def point[A](a: => A): Seq[A] = Seq(a)
def bind[A, B](fa: Seq[A])(f: (A) => Seq[B]): Seq[B] = fa.flatMap(f)
}
Seq is already monadic, so point and bind are easy, empty and plus are monoid operations and Seq is a free monoid
2) Bifoldable[\/]
implicit val bife = new Bifoldable[\/] {
def bifoldMap[A, B, M](fa: \/[A, B])(f: (A) => M)(g: (B) => M)(implicit F: Monoid[M]): M = fa match {
case \/-(r) => g(r)
case -\/(l) => f(l)
}
def bifoldRight[A, B, C](fa: \/[A, B], z: => C)(f: (A, => C) => C)(g: (B, => C) => C): C = fa match {
case \/-(r) => g(r, z)
case -\/(l) => f(l, z)
}
}
Also easy, standard folding, but for type constructors with two parameters.
Now you can use separate:
val seq: Seq[String \/ Int] = List(\/-(10), -\/("wrong"), \/-(22), \/-(1), -\/("exception"))
scala> seq.separate
res2: (Seq[String], Seq[Int]) = (List(wrong, number exception),List(10, 22, 1))
Update
Thanks to Kenji Yoshida, there is a Bitraverse[\/], so you need only MonadPlus.
And a simple solution using foldLeft:
seq.foldLeft((Seq.empty[String], Seq.empty[Int])){ case ((as, ai), either) =>
either match {
case \/-(r) => (as, ai :+ r)
case -\/(l) => (as :+ l, ai)
}
}
I occasionally hit code like this:
val things : List[A \/ B] = ???
val (as, bs) : (List[A], List[B]) = ??? //insert something to do this
or in my current case I want Map[A, B \/ C] => (Map[A, B], Map[A, C])
Is there a nice way to do this in the general case F[A \/ B] with appropriate restrictions on F? It looks vaguely like a variation on the theme of Unzip.
Here's how we deal with this for / but also Either and Validation, and not just for Lists, but other Foldable.
object Uncozip {
implicit val wtf = language.higherKinds
// Typeclass which covers sum types such as \/, Either, Validation
trait Sum2[F[_, _]] {
def cata[A, B, X](a: A ⇒ X, b: B ⇒ X)(fab: F[A, B]): X
}
implicit val sumEither: Sum2[Either] = new Sum2[Either] {
def cata[A, B, X](a: A ⇒ X, b: B ⇒ X)(fab: Either[A, B]): X = {
fab match {
case Left(l) ⇒ a(l)
case Right(r) ⇒ b(r)
}
}
}
implicit val sumEitherz: Sum2[\/] = new Sum2[\/] {
def cata[A, B, X](a: A ⇒ X, b: B ⇒ X)(fab: A \/ B): X = {
fab.fold(a(_), b(_))
}
}
implicit val sumValidation: Sum2[Validation] = new Sum2[Validation] {
def cata[A, B, X](a: A ⇒ X, b: B ⇒ X)(fab: A Validation B): X = {
fab.fold(a(_), b(_))
}
}
abstract class Uncozips[F[_], G[_, _], A, B](fab: F[G[A, B]]) {
def uncozip: (F[A], F[B])
}
implicit def uncozip[F[_]: Foldable, G[_, _], A, B](fab: F[G[A, B]])(implicit g: Sum2[G], mfa: ApplicativePlus[F], mfb: ApplicativePlus[F]): Uncozips[F, G, A, B] = new Uncozips[F, G, A, B](fab) {
def uncozip = {
implicitly[Foldable[F]].foldRight[G[A, B], (F[A], F[B])](fab, (mfa.empty, mfb.empty)) { (l, r) ⇒
g.cata[A, B, (F[A], F[B])]({ (a: A) ⇒ (mfa.plus(mfa.point(a), r._1), r._2) },
{ (b: B) ⇒ (r._1, mfa.plus(mfa.point(b), r._2)) })(l)
}
}
}
}
You can map things in to a list of (Option[A], Option[B]), unzip that list in to two lists, and then unite the resulting lists:
import scalaz._
import Scalaz._
val things: List[String \/ Int] = List("foo".left, 42.right)
val (strs, ints): (List[String], List[Int]) = things.
map { d => (d.swap.toOption, d.toOption) }. // List[(Option[String], Option[Int])]
unzip. // (List[Option[String]], List[Option[Int]])
bimap(_.unite, _.unite) // (List[String], List[Int])
This isn't particularly efficient due to traversing the list three times.
Here is one way (for lists):
val things : List[A \/ B] = ???
val (as, bs) = (things.map(_.swap.toList).join, things.map(_.toList).join)
And for a map:
val things: Map[String, String \/ Int] = ???
val (as, bs) = (things.mapValues(_.swap.toList).filterNot(e => e._2.isEmpty),
things.mapValues(_.toList).filterNot(e => e._2.isEmpty))
I'm having a hard time coming up with a way to generalize this over any F (I believe you would need instances of Monoid and Applicative for F).