I'd like to migrate the following code from Scala 2.12 to 2.13
Given any collection of tuples Coll[(A, B)] and a method f: B => IterableOnce[C], I'd like to produce a Coll[(A, C)] by applying f on the second element of the tuple.
implicit class TuplesOps[A, B, Repr <: Traversable[(A, B)]](val s: TraversableLike[(A, B), Repr]) extends AnyVal {
def flatMapValues[C, That](f: B => TraversableOnce[C])(implicit bf: CanBuildFrom[Repr, (A, C), That]) =
s.flatMap { case (a, b) => f(b).map((a, _))}
}
I know that the collection API has changed in 2.13 and I read : https://docs.scala-lang.org/overviews/core/custom-collection-operations.html
I've tried 2 implementations
First
import scala.collection.{ AbstractView, BuildFrom }
import scala.collection.generic.IsSeq
object Evidence extends App {
class TuplesOps[Repr, S <: IsSeq[Repr]](coll: Repr, seq: S) {
def flatMapValues[B, C, D, That](f: C => IterableOnce[D])(implicit bf: BuildFrom[Repr, (B, D), That], ev: seq.A =:= (B, C)): That = {
val seqOps = seq(coll)
bf.fromSpecific(coll)(new AbstractView[(B, D)] {
override def iterator: Iterator[(B, D)] = {
seqOps.flatMap { x => f(ev(x)._2).map((ev(x)._1, _))}.iterator
}
})
}
}
implicit def TuplesOps[Repr](coll: Repr)(implicit seq: IsSeq[Repr]): TuplesOps[Repr, seq.type] =
new TuplesOps(coll, seq)
List("a"->1, "b"->2).flatMapValues{(x:Int) => Seq.fill(x)("x")}
}
And I have this compilation error :
[error] /home/yamo/projects/perso/coll213/src/main/scala/example/Evidence.scala:22:37: diverging implicit expansion for type scala.collection.BuildFrom[List[(String, Int)],(B, String),That]
[error] starting with method Tuple9 in object Ordering
[error] List("a"->1, "b"->2).flatMapValues{(x:Int) => Seq.fill(x)("x")}
Second
import scala.collection.{ AbstractView, BuildFrom }
import scala.collection.generic.IsSeq
object Aux extends App {
type Aux[Repr, B, C] = IsSeq[Repr] { type A = (B, C)}
class TuplesOps[Repr, B, C, S <: Aux[Repr, B, C]](coll: Repr, seq: S) {
def flatMapValues[D, That](f: C => IterableOnce[D])(implicit bf: BuildFrom[Repr, (B, D), That]): That = {
val seqOps = seq(coll)
bf.fromSpecific(coll)(new AbstractView[(B, D)] {
// same as before
override def iterator: Iterator[(B, D)] = {
seqOps.flatMap { case (b, c) => f(c).map((b, _))}.iterator
}
})
}
}
implicit def TuplesOps[Repr, B, C](coll: Repr)(implicit seq: Aux[Repr, B, C]): TuplesOps[Repr, B, C, seq.type] =
new TuplesOps(coll, seq)
List("a"->1, "b"->2).flatMapValues(Seq.fill(_)("x"))
}
And I have this compilation error
[error] /home/yamo/projects/perso/coll213/src/main/scala/example/Aux.scala:24:24: value flatMapValues is not a member of List[(String, Int)]
[error] List("a"->1, "b"->2).flatMapValues(Seq.fill(_)("x"))
Could you help me to make both solution work, if possible ?
I agree, the Custom Collection instructions aren't what they should be.
I find the Factory method a bit easier than the BuildFrom, but I'm still getting used to either/both of them.
import scala.collection.Factory
implicit
class TuplesOps[A,B,CC[x] <: Iterable[x]](val ts: CC[(A,B)]) {
def flatMapValues[C](f: B => IterableOnce[C]
)(implicit fac: Factory[(A,C), CC[(A,C)]]
): CC[(A,C)] =
ts.flatMap{case (a,b) => f(b).iterator.map(x => (a,x))}
.to[CC[(A,C)]](fac)
}
This passes all my simple (minded) tests. It won't work on an Array or an Iterator but then your original Scala 2.12.x version didn't either (for me) so I figured that was okay.
Related
Note - the operation described below now exists in the standard library as partitionMap but I believe it's still a valid question as to how to achieve more general ends
Question regarding scala 2.13 - how do I consume/construct collections of specific types when adding custom collections operations where I need to restrict the element types of the input collections? e.g. how do I define:
def split[CC[_], A, B](coll: CC[Either[A, B]]): (CC[A], CC[B])
Following the documentation I've managed to achieve this as follows:
import collection.generic.IsIterable
import scala.collection.{BuildFrom, Factory}
class SplitOperation[Repr, S <: IsIterable[Repr]](coll: Repr, itr: S) {
def split[A, B, AS, BS](
implicit bfa: BuildFrom[Repr, A, AS],
bfb: BuildFrom[Repr, B, BS],
ev: itr.A =:= Either[A, B]): (AS, BS) = {
val ops = itr(coll)
val as = bfa.fromSpecific(coll)(ops.iterator.map(ev).collect { case Left(a) => a })
val bs = bfb.fromSpecific(coll)(ops.iterator.map(ev).collect { case Right(b) => b })
(as, bs)
}
}
implicit def SplitOperation[Repr](coll: Repr)(implicit itr: IsIterable[Repr]): SplitOperation[Repr, itr.type] =
new SplitOperation(coll, itr)
However, I need to supply types at the use-site otherwise I get diverging implicit expansion.
scala> List(Left("bah"), Right(1), Left("gah"), Right(2), Right(3))
res1: List[scala.util.Either[String,Int]] = List(Left(bah), Right(1), Left(gah), Right(2), Right(3))
scala> res1.split
^
error: diverging implicit expansion for type scala.collection.BuildFrom[List[scala.util.Either[String,Int]],A,AS]
But the following works:
scala> res1.split[String, Int, List[String], List[Int]]
res4: (List[String], List[Int]) = (List(bah, gah),List(1, 2, 3))
EDIT
class SplitOperation[X, CC[_], S <: IsIterable[CC[X]]](coll: CC[X], itr: S) {
def split[A, B](implicit bfa: BuildFrom[CC[X], A, CC[A]], bfb: BuildFrom[CC[X], B, CC[B]], ev: itr.A =:= Either[A, B]): (CC[A], CC[B]) = {
val ops = itr(coll)
val as = bfa.fromSpecific(coll)(ops.iterator.map(ev).collect { case Left(a) => a })
val bs = bfb.fromSpecific(coll)(ops.iterator.map(ev).collect { case Right(b) => b })
(as, bs)
}
}
implicit def SplitOperation[A, B, CC[_]](coll: CC[Either[A, B]])(implicit itr: IsIterable[CC[Either[A, B]]]): SplitOperation[Either[A, B], CC, itr.type] =
new SplitOperation(coll, itr)
Gives me a slight improvement. Now I only need to provide type parameters A and B at the call site:
scala> l.split[String, Int]
res2: (List[String], List[Int]) = (List(bah, gah),List(1, 2))
This seems to work:
class SplitOperation[A, B, CC[_], S <: IsIterable[CC[Either[A, B]]]](coll: CC[Either[A, B]], itr: S) {
def split(implicit bfa: BuildFrom[CC[Either[A, B]], A, CC[A]], bfb: BuildFrom[CC[Either[A, B]], B, CC[B]], ev: itr.A =:= Either[A, B]): (CC[A], CC[B]) = {
val ops = itr(coll)
val as = bfa.fromSpecific(coll)(ops.iterator.map(ev).collect { case Left(a) => a })
val bs = bfb.fromSpecific(coll)(ops.iterator.map(ev).collect { case Right(b) => b })
(as, bs)
}
}
implicit def SplitOperation[A, B, CC[_]](coll: CC[Either[A, B]])(implicit itr: IsIterable[CC[Either[A, B]]]): SplitOperation[A, B, CC, itr.type] =
new SplitOperation(coll, itr)
In your case you don’t want to abstract over the “kind” of the collection type constructor (CC[_] vs CC[_, _], etc.), you always use the CC[_] kind, so you don’t need to use IsIterable.
I think it is also not necessary to support “Sorted” collections (eg, SortedSet) because there is no Ordering instance for Either, so you don’t need to use BuildFrom.
implicit class SplitOperation[A, B, CC[X] <: IterableOps[X, CC, CC[X]]](coll: CC[Either[A, B]]) {
def split: (CC[A], CC[B]) = {
val as = coll.iterableFactory.from(coll.iterator.collect { case Left(a) => a })
val bs = coll.iterableFactory.from(coll.iterator.collect { case Right(b) => b })
(as, bs)
}
}
https://scastie.scala-lang.org/64QxHwteQN2i3udSxCa3yw
I would like to "compose" two monoids using cats. If there exists a defined Monoid[(A, A) => Int], then I would like to be able to create a Monoid[Preference[A]] using the combine and empty methods of the Monoid[(A, A) => Int]. I am using the term "composing" loosely here because I am not sure that the transform I want to do is accurately called composition.
Here is my current attempt...
import cats._
import cats.implicits._
trait Preference[A] extends Order[A]
object Preference {
def from[A](f: (A, A) => Int): Preference[A] = {
new Preference[A] {
def compare(a1: A, a2: A): Int = {
f(a1, a2)
}
}
}
def monoid[A](implicit ev: Monoid[(A, A) => Int]): Monoid[Preference[A]] = {
new Monoid[Preference[A]] {
def combine(p1: Preference[A], p2: Preference[A]): Preference[A] = {
new Preference[A] {
def compare(a1: A, a2:A): Int = {
ev.combine(p1.compare, p2.compare)(a1, a2)
}
}
}
def empty: Preference[A] = {
from(ev.empty)
}
}
}
}
...this compiles but I would like to know if there is a more idiomatic solution available using cats.
Seems like it should be possible to somehow compose the Monoid[(A,A) => Int] with the from combinator that takes a f:(A, A) => Int and returns a Preference[A] to create a Monoid[Preference[A]] but I can not figure out how to do it.
I have seen this SO post which discusses composing monoids using a product combinator which is not what I want.
I'm not aware of anything built-in into cats directly.
It seems that you have an isomorphism between Preference[A] and (A, A) => Int, and you simply want to transfer the monoid-structure from (A, A) => Int to Preference[A]. This can be expressed generically for arbitrary types A and B:
def fromIsomorphicMonoid[A, B](
forward: A => B,
inverse: B => A
)(implicit aMon: Monoid[A]): Monoid[B] = new Monoid[B] {
def combine(b1: B, b2: B): B =
forward(aMon.combine(inverse(b1), inverse(b2)))
def empty: B = forward(aMon.empty)
}
With this helper method, your monoid in Preference becomes just:
def monoid[A](implicit ev: Monoid[(A, A) => Int]): Monoid[Preference[A]] =
fromIsomorphicMonoid(
from,
(p: Preference[A]) => (x:A, y:A) => p.compare(x, y)
)
Full compilable example (without any dependencies):
trait Monoid[X] {
def empty: X
def combine(x: X, y: X): X
}
trait Order[A] {
def compare(a1: A, a2: A): Int
}
def fromIsomorphicMonoid[A, B](
forward: A => B,
inverse: B => A
)(implicit aMon: Monoid[A]): Monoid[B] = new Monoid[B] {
def combine(b1: B, b2: B): B =
forward(aMon.combine(inverse(b1), inverse(b2)))
def empty: B = forward(aMon.empty)
}
trait Preference[A] extends Order[A]
object Preference {
def from[A](f: (A, A) => Int): Preference[A] = {
new Preference[A] {
def compare(a1: A, a2: A): Int = {
f(a1, a2)
}
}
}
def monoid[A](implicit ev: Monoid[(A, A) => Int])
: Monoid[Preference[A]] = fromIsomorphicMonoid(
from,
(p: Preference[A]) => (x:A, y:A) => p.compare(x, y)
)
}
I occasionally hit code like this:
val things : List[A \/ B] = ???
val (as, bs) : (List[A], List[B]) = ??? //insert something to do this
or in my current case I want Map[A, B \/ C] => (Map[A, B], Map[A, C])
Is there a nice way to do this in the general case F[A \/ B] with appropriate restrictions on F? It looks vaguely like a variation on the theme of Unzip.
Here's how we deal with this for / but also Either and Validation, and not just for Lists, but other Foldable.
object Uncozip {
implicit val wtf = language.higherKinds
// Typeclass which covers sum types such as \/, Either, Validation
trait Sum2[F[_, _]] {
def cata[A, B, X](a: A ⇒ X, b: B ⇒ X)(fab: F[A, B]): X
}
implicit val sumEither: Sum2[Either] = new Sum2[Either] {
def cata[A, B, X](a: A ⇒ X, b: B ⇒ X)(fab: Either[A, B]): X = {
fab match {
case Left(l) ⇒ a(l)
case Right(r) ⇒ b(r)
}
}
}
implicit val sumEitherz: Sum2[\/] = new Sum2[\/] {
def cata[A, B, X](a: A ⇒ X, b: B ⇒ X)(fab: A \/ B): X = {
fab.fold(a(_), b(_))
}
}
implicit val sumValidation: Sum2[Validation] = new Sum2[Validation] {
def cata[A, B, X](a: A ⇒ X, b: B ⇒ X)(fab: A Validation B): X = {
fab.fold(a(_), b(_))
}
}
abstract class Uncozips[F[_], G[_, _], A, B](fab: F[G[A, B]]) {
def uncozip: (F[A], F[B])
}
implicit def uncozip[F[_]: Foldable, G[_, _], A, B](fab: F[G[A, B]])(implicit g: Sum2[G], mfa: ApplicativePlus[F], mfb: ApplicativePlus[F]): Uncozips[F, G, A, B] = new Uncozips[F, G, A, B](fab) {
def uncozip = {
implicitly[Foldable[F]].foldRight[G[A, B], (F[A], F[B])](fab, (mfa.empty, mfb.empty)) { (l, r) ⇒
g.cata[A, B, (F[A], F[B])]({ (a: A) ⇒ (mfa.plus(mfa.point(a), r._1), r._2) },
{ (b: B) ⇒ (r._1, mfa.plus(mfa.point(b), r._2)) })(l)
}
}
}
}
You can map things in to a list of (Option[A], Option[B]), unzip that list in to two lists, and then unite the resulting lists:
import scalaz._
import Scalaz._
val things: List[String \/ Int] = List("foo".left, 42.right)
val (strs, ints): (List[String], List[Int]) = things.
map { d => (d.swap.toOption, d.toOption) }. // List[(Option[String], Option[Int])]
unzip. // (List[Option[String]], List[Option[Int]])
bimap(_.unite, _.unite) // (List[String], List[Int])
This isn't particularly efficient due to traversing the list three times.
Here is one way (for lists):
val things : List[A \/ B] = ???
val (as, bs) = (things.map(_.swap.toList).join, things.map(_.toList).join)
And for a map:
val things: Map[String, String \/ Int] = ???
val (as, bs) = (things.mapValues(_.swap.toList).filterNot(e => e._2.isEmpty),
things.mapValues(_.toList).filterNot(e => e._2.isEmpty))
I'm having a hard time coming up with a way to generalize this over any F (I believe you would need instances of Monoid and Applicative for F).
I am trying to implement a function that would work on types that have a map and a flatMap method. I have already made it for Traversable, but this does not include Future and Option directly. So I have decided to go with my own interface, using a typeclass:
trait CanMap[A, M[_]] {
def map[B](l: M[A])(f: A => B): M[B]
def flatMap[B](l: M[A])(f: A => M[B]): M[B]
}
I already implemented this for Option:
implicit def canmapopt[A] = new CanMap[A, Option] {
def map[B](l: Option[A])(f: A => B): Option[B] = l.map(f)
def flatMap[B](l: Option[A])(f: A => Option[B]): Option[B] = l.flatMap(f)
}
and this one works very well. Now I wanted to implement it for any subtype of Traversable, I tried an implementation very close to the one for Option:
implicit def canmaptrav[A, B, T[B] <: Traversable[B]] = new CanMap[A, T] {
def map[B](l: T[A])(f: A => B): T[B] = l.map(f)
def flatMap[B](l: T[A])(f: A => T[B]): T[B] = l.flatMap(f)
}
but I get the error:
type mismatch; found : Traversable[B] required: T[B] Note: implicit method canmaptrav is not applicable here because it comes after the application point and it lacks an explicit result type
for the return type of l.map. I cannot understand why l.map(f) would return a Traversable and not the specific type T[B]. So I tried to explicitelly put the right type of CanBuildFrom in the context:
implicit def canmaptrav[A, B, T[B] <: Traversable[B]](implicit cbf: CanBuildFrom[T[A], B, T[B]]) = new CanMap[A, T] {
def map[B](l: T[A])(f: A => B): T[B] = l.map(f)
def flatMap[B](l: T[A])(f: A => T[B]): T[B] = l.flatMap(f)
}
The error persists.
Any idea where I went wrong? It might be obvious but I am getting confused with the generic type signatures I guess.
Update: Solution
First of all, as the answers pointed out, CanMap is mostly a Functor/Monad, so if you dare, you can use scalaz to implement this. However, if you are like me and want to try without it, here is the solution, based on the answer by Kipton Barros:
trait CanMap[A, B, M[_]] {
def map(l: M[A])(f: A => B): M[B]
def flatMap(l: M[A])(f: A => M[B]): M[B]
}
implicit def canmapopt[A, B] = new CanMap[A, B, Option] {
def map(l: Option[A])(f: A => B): Option[B] = l.map(f)
def flatMap(l: Option[A])(f: A => Option[B]): Option[B] = l.flatMap(f)
}
implicit def canmaptrav[A, B, M[+_]](implicit bf: CanBuildFrom[M[A], B, M[B]], ev: M[A] => TraversableLike[A, M[A]], eb: M[B] => TraversableLike[B, M[B]]) = new CanMap[A, B, M] {
def map(l: M[A])(f: (A) => B): M[B] = l.map(f)
def flatMap(l: M[A])(f: A => M[B]): M[B] = l.flatMap[B, M[B]] { (a: A) =>
f(a)
}
}
The trick is to use an implicit conversion M[A] => TraversableLike[A, M[A]] instead of trying to subtype Traversable.
The first problem is that there's a lot going on "under the hood" in the Traversable map method. It does a bit of work to return the most specific collection type, which is why you need CanBuildFrom. The second problem is that Option does not implement the Traversable interface, so its map method doesn't take a CanBuildFrom.
Here's the closest I could get,
import scala.collection.generic.CanBuildFrom
import collection.TraversableLike
trait CanMap[A, M[_]] {
def map[B](l: M[A])(f: A => B)(implicit bf: CanBuildFrom[M[A], B, M[B]]): M[B]
}
object Test {
// ugly hack to work around nonexistent CanBuildFrom for Option
implicit def optionBuilder[A, B]: CanBuildFrom[Option[A], B, Option[B]] = null
implicit def canmapopt[A] = new CanMap[A, Option] {
def map[B](l: Option[A])(f: A => B)(implicit bf: CanBuildFrom[Option[A], B, Option[B]]): Option[B] = l.map(f)
}
implicit def canmaptrav[A, M[_]](implicit ev: M[A] => TraversableLike[A, M[A]]) = new CanMap[A, M] {
def map[B](l: M[A])(f: (A) => B)(implicit bf: CanBuildFrom[M[A], B, M[B]]): M[B] = l.map(f)
}
// example usage
def mapper[A, B, M[_]](l: M[A])(f: A => B)(implicit cm: CanMap[A,M], bf: CanBuildFrom[M[A], B, M[B]]) = {
cm.map(l)(f)
}
mapper(List(1,2,3))(_ + 1) // List(2,3,4)
mapper(Some(2): Option[Int])(_ + 1) // Some(3)
// (cast to Option[Int] is needed to find the canmapopt implicit)
}
By the way, the implicit conversion to TraversableLike makes this also work with arrays,
mapper(Array(1,2,3))(_ + 1) // Array(2, 3, 4)
Firstly I tried your two failed attempts and didn't got much out of it. Then I decided to go simple and do my own CanMap implementation. I ended up with this:
def canmaptrav[A] = new CanMap[A, Traversable]{
def map[B](l: Traversable[A])(f: A => B): Traversable[B]= l.map(f)
def flatMap[B](l: Traversable[A])(f: A => Traversable[B]): Traversable[B] = l.flatMap(f)
}
Looking exactly the CanMap[_,Option]. Assuming the subtypes you are looking for is for use cases like this:
canmaptrav[Int].map(List(1,2,3,4))(_ + 1) //> res0: Traversable[Int] = List(2, 3, 4, 5)
canmaptrav[Int].map(Vector(1,2,3,4))(_ + 1) //> res1: Traversable[Int] = Vector(2, 3, 4, 5)
Now, if you want res1 and res0 types to be the concrete types (List, Vector) than the approach will have to indeed rely on the CanBuildFrom from.
BTW, you know that the CanMap is almost the Monad interface, right?
Scalaz already includes these typeclasses and they are called Monad and Functor. A short example:
// map
def foo[F[_] : Functor](xs: F[Int]) = xs.map(_ + 1)
scala> foo(List(1,2,3))
res2: List[Int] = List(2, 3, 4)
// flatMap
def flatten[M[_] : Monad](xs: M[M[Int]]) = xs.flatMap(identity)
scala> flatten(List(List(1,2,3)))
res3: List[Int] = List(1, 2, 3)
edit:
The functor instance for Future could look like this:
implicit object FutureFunctor extends Functor[Future] {
def map[A,B](fa: Future[A])(f: A => B) = fa.map(f)
}
The trait below compiles fine without the #specialized annotations, or without the map method. Otherwise, it will fail with a compilation error, which doesn't make a lot of sense (at least to me):
[error] (compile:compile) scala.tools.nsc.symtab.Types$TypeError: type mismatch;
[error] found : U(in method foreach)(in method foreach)
[error] required: U(in method foreach)(in method foreach)
This is the trait that I'm talking about:
trait Tuple2Traversable[#specialized(Int, Byte) +A, #specialized(Int, Byte) +B] {
def foreach[T](fn: (A, B) => T)
def map[T](fn: (A, B) => T): Traversable[T] = new Traversable[T] {
def foreach[U](f: T => U) {
def composed(a: A, b: B) = f(fn(a, b))
Tuple2Traversable.this.foreach(composed)
}
}
def flatMap[T](fn: (A, B) => Traversable[T]): Traversable[T] = new Traversable[T] {
def foreach[U](f: (T) => U) {
def composed(a: A, b: B) = fn(a, b).foreach(f)
Tuple2Traversable.this.foreach(composed)
}
}
def filter(included: (A, B) => Boolean): Tuple2Traversable[A, B] = new Tuple2Traversable[A, B] {
def foreach[T](fn: (A, B) => T) {
def composed(a: A, b: B) = if (included(a, b)) fn(a, b)
Tuple2Traversable.this.foreach(composed)
}
}
def foldLeft[T](z: T)(fn: (T, A, B) => T): T = {
var current = z
def op(a: A, b: B) {
current = fn(current, a, b)
}
foreach(op)
current
}
def asTraversable = new Traversable[(A, B)] {
def foreach[U](f: ((A, B)) => U) {
def tupled(a: A, b: B) = f((a, b))
Tuple2Traversable.this.foreach(tupled)
}
}
}
I've been staring at this for a while now. Any suggestions on how to solve this would be highly appreciated.
Perhaps I should add that the purpose of this class is to have a Traversable of tuples, without forcing those tuples to ever be created. A Traversable[(A,B)] would accept a Tuple2[A,B] => T as the parameter of foreach. I want my 'traversable' to accept a function (A, B) => T. (Like def fn(a: Int, b: Int) = a + b.)
Looks like some internal compiler bug. I get the same error in scala 2.9.2, but it compiles fine in scala 2.10-RC2