I was playing with cats' Monoids in scala when I see that the monoid operations are extended for Tuples in a natural way:
import cats.Monoid
object mon {
implicit object IntMonoid extends Monoid[Int] {
def combine(a: Int, b: Int) = a*a + b*b
def empty = 0
}
implicit object ListMonoid extends Monoid[List[Int]] {
def combine(a: List[Int], b: List[Int]): List[Int] =
a.zip(b).map(z => z._1 * z._2)
def empty = List(1)
}
def comb[T](a: T, b: T)(implicit m: Monoid[T]) =
m.combine(a, b)
}
val list1 = List(1, 2, 3)
val list2 = List(2, 3, 4)
println(mon.comb(list1, list2)) // outputs: List(2, 6, 12) as expected
val int1 = 2
val int2 = 4
println(mon.comb(int1, int2)) // outputs: 20 as expected
val x = (list1, int1)
val y = (list2, int2)
println(mon.comb(x, y)) // outputs: (List(2, 6, 12),20)
The last output is expected in a 'natural' way, but how does de compiler knows how to do it?
I've been trying to look for it in Cats' source code, but I'm not as experienced in Scala as to be able to know what to look for. I suppose the same methods holds for similar constructions like semigroups.
Your question boils down to how implicit derivation of typeclasses for generic types work; so let's see two examples:
A case where we want to provide an instance no matter what the generic is:
// Similar to the code you had, but without being tied to just List[Int],
// Since in this case the Int part is irrelevant.
implicit def monoidList[A]: Monoid[List[A]] =
new Monoid[List[A]] {
override final val empty: List[A] = Nil
override final def combine(l1: List[A], l2: List[A]): List[A] =
l1 ::: l2
}
A case where we require a proof of the generic type to provide the instance of the complex type:
implicit def optionMonoid[A](implicit aMonoid: Monoid[A]): Monoid[Option[A]] =
new Monoid[Option[A]] {
override final val empty: Option[A] = None
override final def combine(o1: Option[A], o2: Option[A]): Option[A] =
(o1, o2) match {
case (None, None) => None
case (Some(a), None) => Some(a)
case (None, Some(a)) => Some(a)
case (Some(a1), Some(a1)) => Some(aMonoid.combine(a1, a2))
}
}
Thus, you can now imagine how the Monoid[Tuple2[A, B]] of cats works, but just for completeness the code would be like this:
implicit def tuple2Monoid[A, B](implicit aMonoid: Monoid[A], bMonoid: Monoid[B]): Monoid[(A, B)] =
new Monoid[(A, B)] {
override final def empty: (A, B) =
(aMonoid.empty, bMonoid.empty)
override final def combine(t1: (A, B), t2: (A, B)): (A, B) =
(t1, t2) match {
case ((a1, b1), (a2, b2)) => (aMonoid.combine(a1, a2), bMonoid.combine(b1, b2))
}
}
Related
I have multiple map functions running over the same data and I'd like to have them run in a single pass. I'm looking for a generic way to do this.
val fruits: Seq[String] = Seq("apple", "banana", "cherry")
def mapF(s: String): Char = s.head
def reduceF(c1: Char, c2: Char): Char = if(c1 > c2) c1 else c2
def mapG(s: String): Int = s.length
def reduceG(i1: Int, i2: Int): Int = i1 + i2
val largestStartingChar = fruits.map(mapF).reduce(reduceF)
val totalStringLength = fruits.map(mapG).reduce(reduceG)
I'd like to reduce the number of passes over fruits. I can make this generic for two maps and reduces like this:
def productMapFunction[A, B, C](f: A=>B, g: A=>C): A => (B, C) = {
x => (f(x), g(x))
}
def productReduceFunction[T, U](f: (T, T)=>T, g: (U, U) => U):
((T,U), (T,U)) => (T, U) = {
(tu1, tu2) => (f(tu1._1, tu2._1), g(tu1._2, tu2._2))
}
val xMapFG = productMapFunction(mapF, mapG)
val xReduceFG = productReduceFunction(reduceF, reduceG)
val (largestStartingChar2, totalStringLength2) =
fruits.map(xMapFG).reduce(xReduceFG))
I'd like to do this even more generically, with an arbitrary number of map and reduce functions, but I'm not sure how to proceed, or if this is possible.
Following solution uses Cats 2 and custom type MapReduce.
Reducing operation can be specified by function reduce: (O, O) => O
or cats reducer: Semigroup[O].
Multiple MapReduce objects can be combined into one by Apply instance provided by implicit def mapReduceApply[I]
import cats._
import cats.implicits._
trait MapReduce[I, O] {
type R
def reducer: Semigroup[R]
def map: I => R
def mapResult: R => O
def apply(input: Seq[I]): O = mapResult(input.map(map).reduce(reducer.combine))
}
object MapReduce {
def apply[I, O, _R](_reducer: Semigroup[_R], _map: I => _R, _mapResult: _R => O): MapReduce[I, O] =
new MapReduce[I, O] {
override type R = _R
override def reducer = _reducer
override def map = _map
override def mapResult = _mapResult
}
def apply[I, O](map: I => O)(implicit r: Semigroup[O]): MapReduce[I, O] =
MapReduce[I, O, O](r, map, identity)
def apply[I, O](map: I => O, reduce: (O, O) => O): MapReduce[I, O] = {
val reducer = new Semigroup[O] {
override def combine(x: O, y: O): O = reduce(x, y)
}
MapReduce(map)(reducer)
}
implicit def mapReduceApply[I] =
new Apply[({type F[X] = MapReduce[I, X]})#F] {
override def map[A, B](f: MapReduce[I, A])(fn: A => B): MapReduce[I, B] =
MapReduce(f.reducer, f.map, f.mapResult.andThen(fn))
override def ap[A, B](ff: MapReduce[I, (A) => B])(fa: MapReduce[I, A]): MapReduce[I, B] =
MapReduce(ff.reducer product fa.reducer,
i => (ff.map(i), fa.map(i)),
(t: (ff.R, fa.R)) => ff.mapResult(t._1)(fa.mapResult(t._2))
)
}
}
object MultiMapReduce extends App {
val fruits: Seq[String] = Seq("apple", "banana", "cherry")
def mapF(s: String): Char = s.head
def reduceF(c1: Char, c2: Char): Char = if (c1 > c2) c1 else c2
val biggestFirsChar = MapReduce(mapF, reduceF)
val totalChars = MapReduce[String, Int](_.length) // (Semigroup[Int]) reduce by _ + _
def count[A] = MapReduce[A, Int](_ => 1)
val multiMapReduce = (biggestFirsChar, totalChars, count[String]).mapN((_, _, _))
println(multiMapReduce(fruits))
val sum = MapReduce[Double, Double](identity)
val average = (sum, count[Double]).mapN(_ / _)
println(sum(List(1, 2, 3, 4)))
println(average(List(1, 2, 3, 4)))
}
Runnable version is also available on GitHub.
Interesting question!
I don't know of any such implementation in the standard library or even scalaz/cats.
It's not very surprising because if your list is not very large you can just perform map-reduces sequentially and I'm not even sure that overhead of constructing lots of intermediate objects would be smaller than overhead of traversing the list several times.
And if the list is potentially doesn't fit into memory you should be using one of the streaming libraries (fs2/zio-streams/akka-streams)
Although if your input was Iterator instead of List, such functionality would be useful.
There is an interesting article about this problem:
https://softwaremill.com/beautiful-folds-in-scala/
tldr:
Map-reduce workflow could be formalized as follows:
trait Fold[I, O] {
type M
def m: Monoid[M]
def tally: I => M
def summarize: M => O
}
In your case I = List[A], tally = list => list.map(mapF), summarize = list => list.reduce(reduceF) .
To run a map-reduce on a list using an instance of fold you need to run
fold.summarize(fold.tally(list))
You can define combine operation on them:
def combine[I, O1, O2](f1: Fold[I, O1], f2: Fold[I, O2]): Fold[I, (O1, O2)]
Using combine few times would give you what you want:
combine(combine(f1, f2), f3): Fold[I, ((O1, O2), O3)]
I think you're just trying to reinvent transducers. It's been a while since I have used Scala, but there's at least one implementation.
Note - the operation described below now exists in the standard library as partitionMap but I believe it's still a valid question as to how to achieve more general ends
Question regarding scala 2.13 - how do I consume/construct collections of specific types when adding custom collections operations where I need to restrict the element types of the input collections? e.g. how do I define:
def split[CC[_], A, B](coll: CC[Either[A, B]]): (CC[A], CC[B])
Following the documentation I've managed to achieve this as follows:
import collection.generic.IsIterable
import scala.collection.{BuildFrom, Factory}
class SplitOperation[Repr, S <: IsIterable[Repr]](coll: Repr, itr: S) {
def split[A, B, AS, BS](
implicit bfa: BuildFrom[Repr, A, AS],
bfb: BuildFrom[Repr, B, BS],
ev: itr.A =:= Either[A, B]): (AS, BS) = {
val ops = itr(coll)
val as = bfa.fromSpecific(coll)(ops.iterator.map(ev).collect { case Left(a) => a })
val bs = bfb.fromSpecific(coll)(ops.iterator.map(ev).collect { case Right(b) => b })
(as, bs)
}
}
implicit def SplitOperation[Repr](coll: Repr)(implicit itr: IsIterable[Repr]): SplitOperation[Repr, itr.type] =
new SplitOperation(coll, itr)
However, I need to supply types at the use-site otherwise I get diverging implicit expansion.
scala> List(Left("bah"), Right(1), Left("gah"), Right(2), Right(3))
res1: List[scala.util.Either[String,Int]] = List(Left(bah), Right(1), Left(gah), Right(2), Right(3))
scala> res1.split
^
error: diverging implicit expansion for type scala.collection.BuildFrom[List[scala.util.Either[String,Int]],A,AS]
But the following works:
scala> res1.split[String, Int, List[String], List[Int]]
res4: (List[String], List[Int]) = (List(bah, gah),List(1, 2, 3))
EDIT
class SplitOperation[X, CC[_], S <: IsIterable[CC[X]]](coll: CC[X], itr: S) {
def split[A, B](implicit bfa: BuildFrom[CC[X], A, CC[A]], bfb: BuildFrom[CC[X], B, CC[B]], ev: itr.A =:= Either[A, B]): (CC[A], CC[B]) = {
val ops = itr(coll)
val as = bfa.fromSpecific(coll)(ops.iterator.map(ev).collect { case Left(a) => a })
val bs = bfb.fromSpecific(coll)(ops.iterator.map(ev).collect { case Right(b) => b })
(as, bs)
}
}
implicit def SplitOperation[A, B, CC[_]](coll: CC[Either[A, B]])(implicit itr: IsIterable[CC[Either[A, B]]]): SplitOperation[Either[A, B], CC, itr.type] =
new SplitOperation(coll, itr)
Gives me a slight improvement. Now I only need to provide type parameters A and B at the call site:
scala> l.split[String, Int]
res2: (List[String], List[Int]) = (List(bah, gah),List(1, 2))
This seems to work:
class SplitOperation[A, B, CC[_], S <: IsIterable[CC[Either[A, B]]]](coll: CC[Either[A, B]], itr: S) {
def split(implicit bfa: BuildFrom[CC[Either[A, B]], A, CC[A]], bfb: BuildFrom[CC[Either[A, B]], B, CC[B]], ev: itr.A =:= Either[A, B]): (CC[A], CC[B]) = {
val ops = itr(coll)
val as = bfa.fromSpecific(coll)(ops.iterator.map(ev).collect { case Left(a) => a })
val bs = bfb.fromSpecific(coll)(ops.iterator.map(ev).collect { case Right(b) => b })
(as, bs)
}
}
implicit def SplitOperation[A, B, CC[_]](coll: CC[Either[A, B]])(implicit itr: IsIterable[CC[Either[A, B]]]): SplitOperation[A, B, CC, itr.type] =
new SplitOperation(coll, itr)
In your case you don’t want to abstract over the “kind” of the collection type constructor (CC[_] vs CC[_, _], etc.), you always use the CC[_] kind, so you don’t need to use IsIterable.
I think it is also not necessary to support “Sorted” collections (eg, SortedSet) because there is no Ordering instance for Either, so you don’t need to use BuildFrom.
implicit class SplitOperation[A, B, CC[X] <: IterableOps[X, CC, CC[X]]](coll: CC[Either[A, B]]) {
def split: (CC[A], CC[B]) = {
val as = coll.iterableFactory.from(coll.iterator.collect { case Left(a) => a })
val bs = coll.iterableFactory.from(coll.iterator.collect { case Right(b) => b })
(as, bs)
}
}
https://scastie.scala-lang.org/64QxHwteQN2i3udSxCa3yw
Just started learning Scalaz. Here is my code
trait Monoid[A] {
def mappend(a1: A, a2: A): A
def mzero: A
}
object Monoid {
implicit val IntMonoid: Monoid[Int] = new Monoid[Int] {
def mappend(a1: Int, a2: Int): Int = a1 + a2
def mzero: Int = 0
}
implicit val StringMonoid: Monoid[String] = new Monoid[String] {
def mappend(a1: String, a2: String): String = a1 + a2
def mzero: String = ""
}
}
trait MonoidOp[A] {
val F: Monoid[A]
val value: A
def |+|(a2: A): A = F.mappend(value, a2)
}
object MonoidOp{
implicit def toMonoidOp[A: Monoid](a: A): MonoidOp[A] = new MonoidOp[A]{
val F = implicitly[Monoid[A]]
val value = a
}
}
I have defined a function (just for the sake of it)
def addXY[A: Monoid](x: A, y: A): A = x |+| y
I want to lift it so that it could be used using Containers like Option, List, etc. But when I do this
def addXYOptioned = Functor[Option].lift(addXY)
It says error: could not find implicit value for evidence parameter of type scalaz.Monoid[A]
def addOptioned = Functor[Option].lift(addXY)
How to lift such functions?
Your method addXY needs a Monoid[A] but there is no Monoid[A] in scope when used in addXYOptioned, so you also need to add the Monoid constraint to addXYOptioned.
The next problem is that Functor.lift only lifts a function A => B, but we can use Apply.lift2 to lift a function (A, B) => C.
Using the Monoid from Scalaz itself :
import scalaz._, Scalaz._
def addXY[A: Monoid](x: A, y: A): A = x |+| y
def addXYOptioned[A: Monoid] = Apply[Option].lift2(addXY[A] _)
We could generalize addXYOptioned to make it possible to lift addXY into any type constructor with an Apply instance :
def addXYApply[F[_]: Apply, A: Monoid] = Apply[F].lift2(addXY[A] _)
addXYApply[List, Int].apply(List(1,2), List(3,4))
// List[Int] = List(4, 5, 5, 6)
addXYApply[Option, Int].apply(1.some, 2.some)
// Option[Int] = Some(3)
This is a followup to my previous question with an example found on the Internet.
Suppose I define a typeclass Applicative as follows:
trait Functor[T[_]]{
def map[A,B](f:A=>B, ta:T[A]):T[B]
}
trait Applicative[T[_]] extends Functor[T] {
def unit[A](a:A):T[A]
def ap[A,B](tf:T[A=>B], ta:T[A]):T[B]
}
I can define an instance of Applicative for List
object AppList extends Applicative[List] {
def map[A,B](f:A=>B, as:List[A]) = as.map(f)
def unit[A](a: A) = List(a)
def ap[A,B](fs:List[A=>B], as:List[A]) = for(f <- fs; a <- as) yield f(a)
}
For convenience I can define an implicit conversion to add a method <*> to List[A=>B]
implicit def toApplicative[A, B](fs: List[A=>B]) = new {
def <*>(as: List[A]) = AppList.ap(fs, as)
}
Now I can do a cool thing !
zip two lists List[String] and apply f2 to every pair in applicative style
val f2: (String, String) => String = {(first, last) => s"$first $last"}
val firsts = List("a", "b", "c")
val lasts = List("x", "y", "z")
scala> AppList.unit(f2.curried) <*> firsts <*> lasts
res31: List[String] = List(a x, a y, a z, b x, b y, b z, c x, c y, c z)
So far, so good but now I have:
val firstsOpt = Some(firsts)
val lastsOpt = Some(lasts)
I would like to zip firsts and lasts, apply f2, and get Option[List[String]] in applicative style. In other words I need <*> for Option[List[_]]. How can I do it ?
Firstly, you need an instance of applicative for Option:
implicit object AppOption extends Applicative[Option] {
def map[A, B](f: A => B, o: Option[A]) = o.map(f)
def unit[A](a: A): Option[A] = Some(a)
def ap[A, B](of: Option[A => B], oa: Option[A]) = of match {
case Some(f) => oa.map(f)
case None => None
}
}
Then you can also create an applicative instance for the composition of two applicatives (note, based on the Haskell version):
class AppComp[F[_], G[_]](fa: Applicative[F], ga: Applicative[G]) extends Applicative[({ type f[A] = F[G[A]]})#f] {
def map[A, B](f: A => B, a: F[G[A]]): F[G[B]] = fa.map((g: G[A]) => ga.map(f, g), a)
def unit[A](a: A) = fa.unit(ga.unit(a))
def ap[A, B](f: F[G[A => B]], a: F[G[A]]): F[G[B]] = {
val liftg: G[A => B] => (G[A] => G[B]) = gf => (gx => ga.ap(gf, gx))
val ffg: F[G[A] => G[B]] = fa.map(liftg, f)
fa.ap(ffg, a)
}
}
implicit def toComp[F[_], G[_]](implicit fa: Applicative[F], ga: Applicative[G]) = new AppComp[F, G](fa, ga)
Finally you can now do:
val ola = toComp[Option, List]
ola.ap(ola.ap(ola.unit(f2.curried), firstsOpt), lastsOpt)
You could probably also remove some of the noise by generalising <*> to work for any applicative.
I have a trait like the following
private class SeqConverter[T](implicit val conv : Converter[T]) extends Converter[Seq[T]] {
def toJs(x: Seq[T]): JsAny = {
x.foldLeft(JsArray[JsAny]()) { (acc, next) =>
acc.+:(conv.toJs(next))
}
}
def toScala(x: JsAny): Seq[T] = {
val arr = x.asInstanceOf[JsArray[JsObject]]
var lst = List[T]()
arr foreach { x =>
lst = conv.toScala(x) :: lst
}
lst
}
}
I want to have something more like this
private class SeqConverter[T, F <: Seq[T]](implicit val conv : Converter[T]) extends Converter[F] {
def toJs(x: F): JsAny = {
x.foldLeft(JsArray[JsAny]()) { (acc, next) =>
acc.+:(conv.toJs(next))
}
}
def toScala(x: JsAny): Seq[T] = {
//need to construct empty F here to build it
}
}
But the problem is I have no way to get a member of F in order to start preforming construction of it. Is there any way I could get this to exist? It seems like there should be some kind of way to construct an empty member of F so so that I could use +: in order to convert from 1 kind of sequence to another. Does anything like that exist?
UPDATE: if you want to avoid depending on Scalaz, you can define your own type class and instances thereof:
import scala.language.higherKinds
trait Coll[TS[_], T] {
def zero: TS[T]
def append(a: TS[T], b: TS[T]): TS[T]
def point(x: T): TS[T]
}
object Coll {
implicit def listOfTIsColl[T] = new Coll[List, T] {
def zero = Nil
def append(a: List[T], b: List[T]) = a ++ b
def point(x: T) = List(x)
}
implicit def vectorOfTIsColl[T] = new Coll[Vector, T] {
def zero = Vector.empty
def append(a: Vector[T], b: Vector[T]) = a ++ b
def point(x: T) = Vector(x)
}
}
def foo[T, U, TS[_]](xs: TS[T], x: U)(implicit
coll: Coll[TS, T],
ev1: TS[T] <:< Seq[T],
ev2: U =:= T
) = {
(coll.zero, coll.append(coll.zero, coll.point(x)))
}
assert(foo(Vector(1, 2, 3), 4) == (Vector(), Vector(4)))
// foo(Vector(1, 2, 3), 4.4) -- error: Cannot prove that Double =:= Int
// foo(Vector(1, 2, 3), "hello") -- error: Cannot prove that String =:= Int
Note that it's necessary for T and U to be separate type parameters; with def foo[T, TS[_]](xs: TS[T], x: T) ..., you'd be able to use foo as expected, but things like foo(Vector(1, 2, 3), "hello") would work and the type inferencer would infer a type like Vector[Any]. However, with the above, stricter definition of foo, this won't be allowed, which is, at least in idiomatic functional code, desirable.
Scalaz based solution:
Scalaz Monoid and Applicative will help you:
import scalaz._
import Scalaz._
scala> Monoid[List[Int]].zero
res0: List[Int] = List()
scala> Monoid[Vector[Int]].zero
res1: Vector[Int] = Vector()
scala> Monoid[Vector[Int]].append(Vector(1, 2), Vector(3, 4))
res2: Vector[Int] = Vector(1, 2, 3, 4)
and
scala> Applicative[Vector].point(1)
res0: Vector[Int] = Vector(1)
Then, combining Monoid and Applicative will give you all of zero, append, and point/pure:
def foo[T, TS[_], U](xs: TS[T], x: U)(implicit
monoid: Monoid[TS[T]],
applic: Applicative[TS],
ev1: TS[T] <:< Seq[T],
ev2: U =:= T
) = {
(monoid.zero,
monoid.append(monoid.zero, applic.point(x)))
}
then:
> foo(Vector(1, 2, 3), 4)
res1 = (Vector(),Vector(4))
I'm not confident there aren't any conciser solutions, e.g. one that relies on just one type class, but this one seems to work correctly.