I often find myself needing to chain collects where I want to do multiple collects in a single traversal. I also would like to return a "remainder" for things that don't match any of the collects.
For example:
sealed trait Animal
case class Cat(name: String) extends Animal
case class Dog(name: String, age: Int) extends Animal
val animals: List[Animal] =
List(Cat("Bob"), Dog("Spot", 3), Cat("Sally"), Dog("Jim", 11))
// Normal way
val cats: List[Cat] = animals.collect { case c: Cat => c }
val dogAges: List[Int] = animals.collect { case Dog(_, age) => age }
val rem: List[Animal] = Nil // No easy way to create this without repeated code
This really isn't great, it requires multiple iterations and there is no reasonable way to calculate the remainder. I could write a very complicated fold to pull this off, but it would be really nasty.
Instead, I usually opt for mutation which is fairly similar to the logic you would have in a fold:
import scala.collection.mutable.ListBuffer
// Ugly, hide the mutation away
val (cats2, dogsAges2, rem2) = {
// Lose some benefits of type inference
val cs = ListBuffer[Cat]()
val da = ListBuffer[Int]()
val rem = ListBuffer[Animal]()
// Bad separation of concerns, I have to merge all of my functions
animals.foreach {
case c: Cat => cs += c
case Dog(_, age) => da += age
case other => rem += other
}
(cs.toList, da.toList, rem.toList)
}
I don't like this one bit, it has worse type inference and separation of concerns since I have to merge all of the various partial functions. It also requires lots of lines of code.
What I want, are some useful patterns, like a collect that returns the remainder (I grant that partitionMap new in 2.13 does this, but uglier). I also could use some form of pipe or map for operating on parts of tuples. Here are some made up utilities:
implicit class ListSyntax[A](xs: List[A]) {
import scala.collection.mutable.ListBuffer
// Collect and return remainder
// A specialized form of new 2.13 partitionMap
def collectR[B](pf: PartialFunction[A, B]): (List[B], List[A]) = {
val rem = new ListBuffer[A]()
val res = new ListBuffer[B]()
val f = pf.lift
for (elt <- xs) {
f(elt) match {
case Some(r) => res += r
case None => rem += elt
}
}
(res.toList, rem.toList)
}
}
implicit class Tuple2Syntax[A, B](x: Tuple2[A, B]){
def chainR[C](f: B => C): Tuple2[A, C] = x.copy(_2 = f(x._2))
}
Now, I can write this in a way that could be done in a single traversal (with a lazy datastructure) and yet follows functional, immutable practice:
// Relatively pretty, can imagine lazy forms using a single iteration
val (cats3, (dogAges3, rem3)) =
animals.collectR { case c: Cat => c }
.chainR(_.collectR { case Dog(_, age) => age })
My question is, are there patterns like this? It smells like the type of thing that would be in a library like Cats, FS2, or ZIO, but I am not sure what it might be called.
Scastie link of code examples: https://scastie.scala-lang.org/Egz78fnGR6KyqlUTNTv9DQ
I wanted to see just how "nasty" a fold() would be.
val (cats
,dogAges
,rem) = animals.foldRight((List.empty[Cat]
,List.empty[Int]
,List.empty[Animal])) {
case (c:Cat, (cs,ds,rs)) => (c::cs, ds, rs)
case (Dog(_,d),(cs,ds,rs)) => (cs, d::ds, rs)
case (r, (cs,ds,rs)) => (cs, ds, r::rs)
}
Eye of the beholder I suppose.
How about defining a couple utility classes to help you with this?
case class ListCollect[A](list: List[A]) {
def partialCollect[B](f: PartialFunction[A, B]): ChainCollect[List[B], A] = {
val (cs, rem) = list.partition(f.isDefinedAt)
new ChainCollect((cs.map(f), rem))
}
}
case class ChainCollect[A, B](tuple: (A, List[B])) {
def partialCollect[C](f: PartialFunction[B, C]): ChainCollect[(A, List[C]), B] = {
val (cs, rem) = tuple._2.partition(f.isDefinedAt)
ChainCollect(((tuple._1, cs.map(f)), rem))
}
}
ListCollect is just meant to start the chain, and ChainCollect takes the previous remainder (the second element of the tuple) and tries to apply a PartialFunction to it, creating a new ChainCollect object. I'm not particularly fond of the nested tuples this produces, but you may be able to make it look a bit better if you use Shapeless's HLists.
val ((cats, dogs), rem) = ListCollect(animals)
.partialCollect { case c: Cat => c }
.partialCollect { case Dog(_, age) => age }
.tuple
Scastie
Dotty's *: type makes this a bit easier:
opaque type ChainResult[Prev <: Tuple, Rem] = (Prev, List[Rem])
extension [P <: Tuple, R, N](chainRes: ChainResult[P, R]) {
def partialCollect(f: PartialFunction[R, N]): ChainResult[List[N] *: P, R] = {
val (cs, rem) = chainRes._2.partition(f.isDefinedAt)
(cs.map(f) *: chainRes._1, rem)
}
}
This does end up in the output being reversed, but it doesn't have that ugly nesting from my previous approach:
val ((owls, dogs, cats), rem) = (EmptyTuple, animals)
.partialCollect { case c: Cat => c }
.partialCollect { case Dog(_, age) => age }
.partialCollect { case Owl(wisdom) => wisdom }
/* more animals */
case class Owl(wisdom: Double) extends Animal
case class Fly(isAnimal: Boolean) extends Animal
val animals: List[Animal] =
List(Cat("Bob"), Dog("Spot", 3), Cat("Sally"), Dog("Jim", 11), Owl(200), Fly(false))
Scastie
And if you still don't like that, you can always define a few more helper methods to reverse the tuple, add the extension on a List without requiring an EmptyTuple to begin with, etc.
//Add this to the ChainResult extension
def end: Reverse[List[R] *: P] = {
def revHelp[A <: Tuple, R <: Tuple](acc: A, rest: R): RevHelp[A, R] =
rest match {
case EmptyTuple => acc.asInstanceOf[RevHelp[A, R]]
case h *: t => revHelp(h *: acc, t).asInstanceOf[RevHelp[A, R]]
}
revHelp(EmptyTuple, chainRes._2 *: chainRes._1)
}
//Helpful types for safety
type Reverse[T <: Tuple] = RevHelp[EmptyTuple, T]
type RevHelp[A <: Tuple, R <: Tuple] <: Tuple = R match {
case EmptyTuple => A
case h *: t => RevHelp[h *: A, t]
}
And now you can do this:
val (cats, dogs, owls, rem) = (EmptyTuple, animals)
.partialCollect { case c: Cat => c }
.partialCollect { case Dog(_, age) => age }
.partialCollect { case Owl(wisdom) => wisdom }
.end
Scastie
Since you mentioned cats, I would also add solution using foldMap:
sealed trait Animal
case class Cat(name: String) extends Animal
case class Dog(name: String) extends Animal
case class Snake(name: String) extends Animal
val animals: List[Animal] = List(Cat("Bob"), Dog("Spot"), Cat("Sally"), Dog("Jim"), Snake("Billy"))
val map = animals.foldMap{ //Map(other -> List(Snake(Billy)), cats -> List(Cat(Bob), Cat(Sally)), dogs -> List(Dog(Spot), Dog(Jim)))
case d: Dog => Map("dogs" -> List(d))
case c: Cat => Map("cats" -> List(c))
case o => Map("other" -> List(o))
}
val tuples = animals.foldMap{ //(List(Dog(Spot), Dog(Jim)),List(Cat(Bob), Cat(Sally)),List(Snake(Billy)))
case d: Dog => (List(d), Nil, Nil)
case c: Cat => (Nil, List(c), Nil)
case o => (Nil, Nil, List(o))
}
Arguably it's more succinct than fold version, but it has to combine partial results using monoids, so it won't be as performant.
This code is dividing a list into three sets, so the natural way to do this is to use partition twice:
val (cats, notCat) = animals.partitionMap{
case c: Cat => Left(c)
case x => Right(x)
}
val (dogAges, rem) = notCat.partitionMap {
case Dog(_, age) => Left(age)
case x => Right(x)
}
A helper method can simplify this
def partitionCollect[T, U](list: List[T])(pf: PartialFunction[T, U]): (List[U], List[T]) =
list.partitionMap {
case t if pf.isDefinedAt(t) => Left(pf(t))
case x => Right(x)
}
val (cats, notCat) = partitionCollect(animals) { case c: Cat => c }
val (dogAges, rem) = partitionCollect(notCat) { case Dog(_, age) => age }
This is clearly extensible to more categories, with the slight irritation of having to invent temporary variable names (which could be overcome by explicit n-way partition methods)
Is there a way to bind a value of Either[L1, R1] to a function of type R1 => Either[L2, R2] and get a value of Either[L1 | L2, R2] so individual functions can declare and potentially return their errors and consumers of a monadic pipeline of these functions can cleanly handle all possible errors in an exhaustive, type-safe way?
Edit
Here's an example...
sealed trait IncrementError
case object MaximumValueReached extends IncrementError
def increment(n: Int): Either[IncrementError, Int] = n match {
case Integer.MAX_VALUE => Left(MaximumValueReached)
case n => Right(n + 1)
}
sealed trait DecrementError
case object MinimumValueReached extends DecrementError
def decrement(n: Int): Either[DecrementError, Int] = n match {
case Integer.MIN_VALUE => Left(MinimumValueReached)
case n => Right(n - 1)
}
for {
n <- increment(0).right
n <- decrement(n).right
} yield n // scala.util.Either[Object, Int] = Right(0)
With that return type I'm not able to do exhaustive error handling. I'm curious if there exists a way to do this using standard Scala Either or if there exists something in a library like scalaz which supports this behavior. I'd like to be able to handle errors like this...
val n = for {
n <- increment(0).right
n <- decrement(n).right
} yield n // scala.util.Either[IncrementError | DecrementError, Int]
match n {
case Left(MaximumValueReached) => println("Maximum value reached!")
case Left(MinimumValueReached) => println("Minimum value reached!")
case Right(_) => println("Success!")
}
I would do following:
/**
* Created by alex on 10/3/16.
*/
object Temp{
sealed trait IncrementError
case object MaximumValueReached extends IncrementError
sealed trait DecrementError
case object MinimumValueReached extends DecrementError
type MyResult = Either[Either[IncrementError, DecrementError], Int]
def increment(n: Int): MyResult = n match {
case Integer.MAX_VALUE => Left(Left(MaximumValueReached))
case n => Right(n + 1)
}
def decrement(n: Int): MyResult = n match {
case Integer.MIN_VALUE => Left(Right(MinimumValueReached))
case n => Right(n - 1)
}
def main(args:Array[String]) = {
val result = for {
k <- increment(0).right
n <- decrement(k).right
} yield n
result match {
case Left(Left(MaximumValueReached)) => println("Maximum value reached!")
case Left(Right(MinimumValueReached)) => println("Minimum value reached!")
case Right(_) => println("Success!")
}
}
}
Honestly I don't like it but it works for the case if you don't want to have IncrementError and DecrementError to inherit from some ancestor trait for some reasons.
I had tried to implement a foldLeft on a LinkedList with this code, one curried foldLeft2, and one not, foldLeft:
sealed trait LinkedList[+E] {
#tailrec
final def foldLeft[A](accumulator: A, f: (A, E) => A): A = {
this match {
case Node(h, t) => {
val current = f(accumulator, h)
t.foldLeft(current, f)
}
case Empty => accumulator
}
}
#tailrec
final def foldLeft2[A](accumulator: A)(f: (A, E) => A): A = {
this match {
case Node(h, t) => {
val current = f(accumulator, h)
t.foldLeft2(current)(f)
}
case Empty => accumulator
}
}
}
But when I use foldLeft, it seems I need to declare the type for accumulator and item, but for foldLeft2, I don't. Can someone explain why that is?
class LinkedListSpecification extends Specification {
"linked list" should {
"foldLeft correctly" in {
val original = LinkedList(1,2,3,4)
original.foldLeft(0, (acc: Int, item: Int) => acc + item) === 10
}
}
"linked list" should {
"foldLeft2 correctly" in {
val original = LinkedList(1,2,3,4)
original.foldLeft2(0)((acc, item) => acc + item) === 10
}
}
}
This is because the type inference in Scala works left-to-right across parameter lists.
Thus in the second version foldLeft2 it is able to infer the type A as Int before it continues to the next parameter list where it now expects a function (Int,E)=>Int.
While in the first version foldLeft it is trying to infer A at the same time by both parameters (accumulator and f). It complains about the anonymous function you are passing to it because it hasn't inferred type A yet.
Here is the code
class Result[A] {
def map[B](f: (Result[A] => Result[B]), xResult: Result[A]) = {
xResult match {
case Success(x) => Success(f(xResult))
case Failure(errs) => Failure(errs)
}
}
def apply[B](fResult: Result[A], xResult: Result[B]) = {
(fResult, xResult) match {
case (Success(f), Success(x)) => Success((f, x))
case (Failure(errs), Success(a)) => Failure(errs)
case (Success(a), Failure(errs)) => Failure(errs)
case (Failure(errs), Failure(errs2)) => Failure(List(errs, errs2))
}
}
}
case class Success[A](a: A) extends Result[A] {}
case class Failure[A](a: A) extends Result[A] {}
def createCustomerId(id: Int) = {
if (id > 0)
Success(id)
else
Failure("CustomerId must be positive")
}
Here are the questions
1) result from type inference from method createCustomer is like this
Product with Serializable with Result[_ >: Int with String]
I dont have trouble with the "Product with Serializable" stuff, the thing im wondering is how to do something meaningfull with the result, and just out of curiosity how to initialize a type like that.
suppose i have another method as shown below
def createCustomer(customerId: Result[_ >: Int with String], email: Result[String]) = {
how can i read/do something with "customerId" argument
}
how to initialize a type like that
val x: Result[Int with String] = ???
notes:
i know my Result class doesnt have any generic "getter"
I know that perhaps things could be easier in method "createCustomerId" if when id > 0 i returned Success(id.ToString), but i really want to know whats going on with this lower type and how i could take advantage (if possible) in the future
The problem is that your type definition states that both Success and Failure take the same argument type. However, you actually want to put some other type of argument into Failure while still keeping covariance. Here is the simplest fix:
class Result[A] {
def map[B](f: (Result[A] => Result[B]), xResult: Result[A]) = {
xResult match {
case Success(x) => Success(f(xResult))
case Failure(errs) => Failure(errs)
}
}
def apply[B](fResult: Result[A], xResult: Result[B]) = {
(fResult, xResult) match {
case (Success(f), Success(x)) => Success((f, x))
case (Failure(errs), Success(a)) => Failure(errs)
case (Success(a), Failure(errs)) => Failure(errs)
case (Failure(errs), Failure(errs2)) => Failure(List(errs, errs2))
}
}
}
case class Success[A](a: A) extends Result[A] {}
case class Failure[A, B](a: B) extends Result[A] {}
object TestResult extends App {
def createCustomerId(id: Int): Result[Int] = {
if (id > 0)
Success(id)
else
Failure("CustomerId must be positive")
}
println(createCustomerId(0))
println(createCustomerId(1))
}
prints:
Failure(CustomerId must be positive)
Success(1)
If you don't annotate return type for createCustomerId with Result[Int] compiler will infer Result[_ <: Int] with Product with Serializable which is probably not ideal but not too bad.
So I got something like this:
abstract class Term
case class App(f:Term,x:Term) extends Term
case class Var(s:String) extends Term
case class Amb(a:Term, b:Term) extends Term //ambiguity
And a Term may look like this:
App(Var(f),Amb(Var(x),Amb(Var(y),Var(z))))
So what I need is all variations that are indicated by the Amb class.
This is used to represent a ambiguous parse forest and I want to type check each possible variation and select the right one.
In this example I would need:
App(Var(f),Var(x))
App(Var(f),Var(y))
App(Var(f),Var(z))
Whats the best way to create these variations in scala?
Efficiency would be nice, but is not really requirement.
If possible I like to refrain from using reflection.
Scala provides pattern matching solve these kinds of problems. A solution would look like:
def matcher(term: Term): List[Term] = {
term match {
case Amb(a, b) => matcher(a) ++ matcher(b)
case App(a, b) => for { va <- matcher(a); vb <- matcher(b) } yield App(va, vb)
case v: Var => List(v)
}
}
You can do this pretty cleanly with a recursive function that traverses the tree and expands ambiguities:
sealed trait Term
case class App(f: Term, x: Term) extends Term
case class Var(s: String) extends Term
case class Amb(a: Term, b: Term) extends Term
def det(term: Term): Stream[Term] = term match {
case v: Var => Stream(v)
case App(f, x) => det(f).flatMap(detf => det(x).map(App(detf, _)))
case Amb(a, b) => det(a) ++ det(b)
}
Note that I'm using a sealed trait instead of an abstract class in order to take advantage of the compiler's ability to check exhaustivity.
It works as expected:
scala> val app = App(Var("f"), Amb(Var("x"), Amb(Var("y"), Var("z"))))
app: App = App(Var(f),Amb(Var(x),Amb(Var(y),Var(z))))
scala> det(app) foreach println
App(Var(f),Var(x))
App(Var(f),Var(y))
App(Var(f),Var(z))
If you can change the Term API, you could more or less equivalently add a def det: Stream[Term] method there.
Since my abstract syntax is fairly large (and I have multiple) and I tried my luck with Kiama.
So here is the version Travis Brown and Mark posted with Kiama.
Its not pretty, but I hope it works. Comments are welcome.
def disambiguateRule: Strategy = rule {
case Amb(a: Term, b: Term) =>
rewrite(disambiguateRule)(a).asInstanceOf[List[_]] ++
rewrite(disambiguateRule)(b).asInstanceOf[List[_]]
case x =>
val ch = getChildren(x)
if(ch.isEmpty) {
List(x)
}
else {
val chdis = ch.map({ rewrite(disambiguateRule)(_) }) // get all disambiguate children
//create all combinations of the disambiguated children
val p = combinations(chdis.asInstanceOf[List[List[AnyRef]]])
//use dup from Kiama to recreate the term with every combination
val xs = for { newchildren <- p } yield dup(x.asInstanceOf[Product], newchildren.toArray)
xs
}
}
def combinations(ll: List[List[AnyRef]]): List[List[AnyRef]] = ll match {
case Nil => Nil
case x :: Nil => x.map { List(_) }
case x :: xs => combinations(xs).flatMap({ ys => x.map({ xx => xx :: ys }) })
}
def getChildren(x: Any): List[Any] = {
val l = new ListBuffer[Any]()
all(queryf {
case a => l += a
})(x)
l.toList
}