Related
How to emulate following behavior in Scala? i.e. keep folding while some certain conditions on the accumulator are met.
def foldLeftWhile[B](z: B, p: B => Boolean)(op: (B, A) => B): B
For example
scala> val seq = Seq(1, 2, 3, 4)
seq: Seq[Int] = List(1, 2, 3, 4)
scala> seq.foldLeftWhile(0, _ < 3) { (acc, e) => acc + e }
res0: Int = 1
scala> seq.foldLeftWhile(0, _ < 7) { (acc, e) => acc + e }
res1: Int = 6
UPDATES:
Based on #Dima answer, I realized that my intention was a little bit side-effectful. So I made it synchronized with takeWhile, i.e. there would be no advancement if the predicate does not match. And add some more examples to make it clearer. (Note: that will not work with Iterators)
First, note that your example seems wrong. If I understand correctly what you describe, the result should be 1 (the last value on which the predicate _ < 3 was satisfied), not 6
The simplest way to do this is using a return statement, which is very frowned upon in scala, but I thought, I'd mention it for the sake of completeness.
def foldLeftWhile[A, B](seq: Seq[A], z: B, p: B => Boolean)(op: (B, A) => B): B = foldLeft(z) { case (b, a) =>
val result = op(b, a)
if(!p(result)) return b
result
}
Since we want to avoid using return, scanLeft might be a possibility:
seq.toStream.scanLeft(z)(op).takeWhile(p).last
This is a little wasteful, because it accumulates all (matching) results.
You could use iterator instead of toStream to avoid that, but Iterator does not have .last for some reason, so, you'd have to scan through it an extra time explicitly:
seq.iterator.scanLeft(z)(op).takeWhile(p).foldLeft(z) { case (_, b) => b }
It is pretty straightforward to define what you want in scala. You can define an implicit class which will add your function to any TraversableOnce (that includes Seq).
implicit class FoldLeftWhile[A](trav: TraversableOnce[A]) {
def foldLeftWhile[B](init: B)(where: B => Boolean)(op: (B, A) => B): B = {
trav.foldLeft(init)((acc, next) => if (where(acc)) op(acc, next) else acc)
}
}
Seq(1,2,3,4).foldLeftWhile(0)(_ < 3)((acc, e) => acc + e)
Update, since the question was modified:
implicit class FoldLeftWhile[A](trav: TraversableOnce[A]) {
def foldLeftWhile[B](init: B)(where: B => Boolean)(op: (B, A) => B): B = {
trav.foldLeft((init, false))((a,b) => if (a._2) a else {
val r = op(a._1, b)
if (where(r)) (op(a._1, b), false) else (a._1, true)
})._1
}
}
Note that I split your (z: B, p: B => Boolean) into two higher-order functions. That's just a personal scala style preference.
What about this:
def foldLeftWhile[A, B](z: B, xs: Seq[A], p: B => Boolean)(op: (B, A) => B): B = {
def go(acc: B, l: Seq[A]): B = l match {
case h +: t =>
val nacc = op(acc, h)
if(p(nacc)) go(op(nacc, h), t) else nacc
case _ => acc
}
go(z, xs)
}
val a = Seq(1,2,3,4,5,6)
val r = foldLeftWhile(0, a, (x: Int) => x <= 3)(_ + _)
println(s"$r")
Iterate recursively on the collection while the predicate is true, and then return the accumulator.
You cand try it on scalafiddle
After a while I received a lot of good looking answers. So, I combined them to this single post
a very concise solution by #Dima
implicit class FoldLeftWhile[A](seq: Seq[A]) {
def foldLeftWhile[B](z: B)(p: B => Boolean)(op: (B, A) => B): B = {
seq.toStream.scanLeft(z)(op).takeWhile(p).lastOption.getOrElse(z)
}
}
by #ElBaulP (I modified a little bit to match comment by #Dima)
implicit class FoldLeftWhile[A](seq: Seq[A]) {
def foldLeftWhile[B](z: B)(p: B => Boolean)(op: (B, A) => B): B = {
#tailrec
def foldLeftInternal(acc: B, seq: Seq[A]): B = seq match {
case x :: _ =>
val newAcc = op(acc, x)
if (p(newAcc))
foldLeftInternal(newAcc, seq.tail)
else
acc
case _ => acc
}
foldLeftInternal(z, seq)
}
}
Answer by me (involving side effects)
implicit class FoldLeftWhile[A](seq: Seq[A]) {
def foldLeftWhile[B](z: B)(p: B => Boolean)(op: (B, A) => B): B = {
var accumulator = z
seq
.map { e =>
accumulator = op(accumulator, e)
accumulator -> e
}
.takeWhile { case (acc, _) =>
p(acc)
}
.lastOption
.map { case (acc, _) =>
acc
}
.getOrElse(z)
}
}
Fist exemple: predicate for each element
First you can use inner tail recursive function
implicit class TravExt[A](seq: TraversableOnce[A]) {
def foldLeftWhile[B](z: B, f: A => Boolean)(op: (A, B) => B): B = {
#tailrec
def rec(trav: TraversableOnce[A], z: B): B = trav match {
case head :: tail if f(head) => rec(tail, op(head, z))
case _ => z
}
rec(seq, z)
}
}
Or short version
implicit class TravExt[A](seq: TraversableOnce[A]) {
#tailrec
final def foldLeftWhile[B](z: B, f: A => Boolean)(op: (A, B) => B): B = seq match {
case head :: tail if f(head) => tail.foldLeftWhile(op(head, z), f)(op)
case _ => z
}
}
Then use it
val a = List(1, 2, 3, 4, 5, 6).foldLeftWhile(0, _ < 3)(_ + _)
//a == 3
Second example: for accumulator value:
implicit class TravExt[A](seq: TraversableOnce[A]) {
def foldLeftWhile[B](z: B, f: A => Boolean)(op: (A, B) => B): B = {
#tailrec
def rec(trav: TraversableOnce[A], z: B): B = trav match {
case _ if !f(z) => z
case head :: tail => rec(tail, op(head, z))
case _ => z
}
rec(seq, z)
}
}
Or short version
implicit class TravExt[A](seq: TraversableOnce[A]) {
#tailrec
final def foldLeftWhile[B](z: B, f: A => Boolean)(op: (A, B) => B): B = seq match {
case _ if !f(z) => z
case head :: tail => tail.foldLeftWhile(op(head, z), f)(op)
case _ => z
}
}
Simply use a branch condition on the accumulator:
seq.foldLeft(0, _ < 3) { (acc, e) => if (acc < 3) acc + e else acc}
However you will run every entry of the sequence.
How is it easier to implement function that find and immutable remove the first occurrence in Scala collection:
case class A(a: Int, b: Int)
val s = Seq(A(1,5), A(4,6), A(2,3), A(5,1), A(2,7))
val (s1, r) = s.findAndRemove(_.a == 2)
Result: s1 = Seq(A(1,5), A(4,6), A(5,1), A(2,7)) , r = Some(A(2,3))
It finds the first element that match, and keeps order. It can be improved with List instead of Seq.
case class A(a: Int, b: Int)
val s = Seq(A(1,5), A(4,6), A(2,3), A(5,1), A(2,7))
val (s1, r) = s.findAndRemove(_.a == 2)
println(s1)
println(r)
implicit class SeqOps[T](s:Seq[T]) {
def findAndRemove(f:T => Boolean):(Seq[T], Option[T]) = {
s.foldLeft((Seq.empty[T], Option.empty[T])) {
case ((l, None), elem) => if(f(elem)) (l, Option(elem)) else (l :+ elem, None)
case ((l, x), elem) => (l :+ elem, x)
}
}
}
Yeah, a little late to the party, but I thought I'd throw this in.
Minimum invocations of the predicate.
Works with most popular collection types: Seq, List, Array, Vector. Even Set and Map (but for those the collection has no order to preserve and there's no telling which element the predicate will find first). Doesn't work for Iterator or String.
-
import scala.collection.generic.CanBuildFrom
import scala.language.higherKinds
implicit class CollectionOps[U, C[_]](xs :C[U]) {
def findAndRemove(p :U=>Boolean
)(implicit bf :CanBuildFrom[C[U], U, C[U]]
,ev :C[U] => collection.TraversableLike[U, C[U]]
) :(C[U], Option[U]) = {
val (before, after) = xs.span(!p(_))
before ++ after.drop(1) -> after.headOption
}
}
usage:
case class A(a: Int, b: Int)
val (as, a) = Seq(A(1,5), A(4,6), A(2,3), A(5,1), A(2,7)).findAndRemove(_.a==2)
//as: Seq[A] = List(A(1,5), A(4,6), A(5,1), A(2,7))
//a: Option[A] = Some(A(2,3))
val (cs, c) = Array('g','t','e','y','b','e').findAndRemove(_<'f')
//cs: Array[Char] = Array(g, t, y, b, e)
//c: Option[Char] = Some(e)
val (ns, n) = Stream.from(9).findAndRemove(_ > 10)
//ns: Stream[Int] = Stream(9, ?)
//n: Option[Int] = Some(11)
ns.take(5).toList //List[Int] = List(9, 10, 12, 13, 14)
Try something like this
def findAndRemove(as: Seq[A])(fn: A => Boolean): (Seq[A], Option[A]) = {
val index = as.indexWhere(fn)
if(index == -1) as -> None
else as.patch(index, Nil, 1) -> as.lift(index)
}
val (s1, r) = findAndRemove(s)(_.a == 2)
My version:
def findAndRemove(s:Seq[A])(p:A => Boolean):(Seq[A], Option[A])={
val i = s.indexWhere(p)
if(i > 0){
val (l1, l2) = s.splitAt(i)
(l1++l2.tail, Some(l2.head))
}else{
(s, None)
}
}
To use guava Table and Multiset in scala? are there already different concenpts in scala instead of importing guava library for this usage?
You could use Map[(R, C), V] instead of Table<R, C, V> and Map[T, Int] instead of Multiset<T>. You could also add helper methods to Map[T, Int] like this:
implicit class Multiset[T](val m: Map[T, Int]) extends AnyVal {
def setAdd(e: T, i: Int = 1) = {
val cnt = m.getOrElse(e, 0) + i
if (cnt <= 0) m - e
else m.updated(e, cnt)
}
def setRemove(e: T, i: Int = 1) = setAdd(e, -i)
def count(e: T) = m.getOrElse(e, 0)
}
val m = Map('a -> 5)
m setAdd 'a
// Map('a -> 6)
m setAdd 'b
// Map('a -> 5, 'b -> 1)
m setAdd ('b, 10)
// Map('a -> 5, 'b -> 10)
m setRemove 'a
// Map('a -> 4)
m setRemove ('a, 6)
// Map()
m count 'b
// 0
(m setAdd 'a) count 'a
// 6
This is a first naive implementation using Map[(R, C), V] as a delegated collection.
rowsand columns are used as indexes for the m: Map.
package utils.collections
import utils.collections.Table.Cell
class Table[R, C, V](val m: Map[(R, C), V], val rows: Map[R, List[(R, C)]], val columns: Map[C, List[(R, C)]]) {
def containsValue(value: V): Boolean = m.values.exists(_.equals(value))
def values(): List[V] = m.values.toList
def get(rowKey: R, columnKey: C): Option[V] = m.get(rowKey, columnKey)
def apply(rowKey: R, columnKey: C): V = m.apply((rowKey, columnKey))
def cellSet(): Set[Cell[R, C, V]] = m.map { case ((r, c), v) => Cell(r, c, v) }.toSet
def contains(rowKey: R, columnKey: C): Boolean = m.contains((rowKey, columnKey))
def put(rowKey: R, columnKey: C, value: V): Table[R, C, V] = {
val keys: (R, C) = (rowKey, columnKey)
new Table(
m = m + ((keys, value)),
rows = rows + ((rowKey, keys::rows.getOrElse(rowKey, List.empty))),
columns = columns + ((columnKey, keys::columns.getOrElse(columnKey, List.empty)))
)
}
def putAll(table: Table[_ <: R, _ <: C, _ <: V]): Table[R, C, V] = Table(m.++(xs = table.m))
def remove(rowKey: R, columnKey: C): Table[R, C, V] = {
val keys: (R, C) = (rowKey, columnKey)
val updatedRows: Map[R, List[(R, C)]] = rows.get(rowKey) match {
case Some(keysWithRow) if keysWithRow.diff(List(keys)).nonEmpty => rows + ((rowKey, keysWithRow.diff(List(keys))))
case _ => rows - rowKey
}
val updatedColumns: Map[C, List[(R, C)]] = columns.get(columnKey) match {
case Some(keysWithColumn) if keysWithColumn.diff(List(keys)).nonEmpty => columns + ((columnKey, keysWithColumn.diff(List(keys))))
case _ => columns - columnKey
}
new Table(
m = m - keys,
rows = updatedRows,
columns = updatedColumns
)
}
def row(rowKey: R): Map[C, V] = m.filterKeys(k => rows.get(rowKey).exists(_.equals(k))).map { case ((_, c), v) => (c, v) }
def containsRow(rowKey: R): Boolean = rows.exists(_.equals(rowKey))
def rowMap(): Map[R, Map[C, V]] = m.groupBy { case ((r, _), _) => r }.map { case (r, subMap) => (r, subMap.map { case ((_, c), v) => (c, v) }) }
def rowKeySet(): Set[R] = rows.keySet
def column(columnKey: C): Map[R, V] = m.filterKeys(k => columns.get(columnKey).exists(_.equals(k))).map { case ((r, _), v) => (r, v) }
def containsColumn(columnKey: C): Boolean = columns.exists(_.equals(columnKey))
def columnMap(): Map[C, Map[R, V]] = m.groupBy { case ((_, c), _) => c }.map { case (c, subMap) => (c, subMap.map { case ((r, _), v) => (r, v) }) }
def columnKeySet(): Set[C] = columns.keySet
def size(): Int = m.size
def isEmpty: Boolean = m.isEmpty
}
object Table {
case class Cell[R, C, V](rowKey: R, columnKey: C, value: V)
def empty[R, C, V] = new Table[R, C, V](Map.empty, Map.empty, Map.empty)
def apply[R, C, V](m: Map[(R, C), V]): Table[R, C, V] = {
val rows: Map[R, List[(R, C)]] = m.keys.groupBy { case (r, c) => r }.map { case (r, keys) => r -> keys.toList }
val columns: Map[C, List[(R, C)]] = m.keys.groupBy { case (r, c) => c }.map { case (c, keys) => c -> keys.toList }
new Table[R, C, V](m, rows, columns)
}
}
I am trying to fix the following issue:
I have a Future[Map[A, B]]. For all B, I need to apply a method that convert B to a Future[C] and I want to give back a Future[Map[A, C]]
Here is the code I have so far:
def getClients(clientIds: Seq[Int]): Future[Map[Int, ClientData]] = {
def getClientData(clientInfo: ClientInfo): Future[ClientData] =
clientInfo match {
case ValidInfo(info) => getData(info)
case _ => throw new Exception
}
client.getClients(clientIds) map {
_.toMap map {
case (clientId: Int, clientInfo: ClientInfo) =>
getClientData(clientInfo) map {
clientData => (clientId, clientData)
}
}
}
}
This code is wrong as it returns a Iterable[Future[(Int, ClientData)]]
For info getClients is a thrift method that returns Future[Map[A, B]] where the Map is mutable, so I need to convert it to an immutable map first with toMap.
Thank you in advance for your help!
scala> def f: Future[Map[String, Future[Int]]] = ???
f: Future[Map[String,Future[Int]]]
scala> def x = for {
| m <- f
| i = m.map{ case (k, fv) => fv.map{ k -> _ } }
| l <- Future.sequence(i)
| } yield l.toMap
x: Future[Map[String,Int]]
Step by step:
Convert Future[Map[A, Future[B]]] to Future[Iterable[Future[(A, B)]]]:
scala> def x1 = f.map{ _.map{ case (k, fv) => fv.map{ k -> _ } } }
x1: Future[Iterable[Future[(String, Int)]]]
Convert Iterable[Future[(A, B)]] to Future[Iterable[(A, B)]] and flatten Future[Future[...]] using flatMap:
scala> def x2 = x1.flatMap{ Future.sequence(_) }
x2: Future[immutable.Iterable[(String, Int)]]
Convert Iterable[(A, B)] to Map[A, B]:
scala> def x = x2.map{ _.toMap }
x: Future[Map[String,Int]]
For com.twitter.util.Future you should use collect instead of sequence and toSeq before collect since it accepts Seq:
def x = for {
m <- f
i = m.map{ case (k, fv) => fv.map{ k -> _ } }
l <- Future.collect(i.toSeq)
} yield l.toMap
This inspiration for this question came when I tried to answer this one.
Say you have a sequence of data (may be from a CSV file for instance). groupBy can be used to analyze certain aspect of the data, grouping by column or a combination of columns. For instance:
val groups0: Map[String, Array[String]] =
seq.groupBy(row => row(0) + "-" + row(4))
If I then want to create sub-groups within the groups I can do
val groups1: Map[String, Map[String, Array[String]]] =
groups0.mapValues(row => row.groupBy(_(1))
If I want to do this more one time it gets really cumbersome:
val groups2 =
groups1.mapValues(groups => groups.mapValues(row => row.groupBy(_(2)))
So here is my question given an arbitrary nesting of Map[K0, Map[K1, ..., Map[Kn, V]]], how do you write a mapValues function that takes a f: (V) => B and applies to the innermost V to return a Map[K0, Map[K1, ..., Map[Kn, B]]]?
My first instinct said that handling arbitrary nesting in a type-safe way would be impossible, but it seems that it IS possible if you define a few implicits that tell the compiler how to do it.
Essentially, the "simple" mapper tells it how to handle the plain non-nested case, while "wrappedMapper" tells it how to drill down through one Map layer:
// trait to tell us how to map inside of a container.
trait CanMapInner[WrappedV, WrappedB,V,B] {
def mapInner(in: WrappedV, f: V => B): WrappedB
}
// simple base case (no nesting involved).
implicit def getSimpleMapper[V,B] = new CanMapInner[V,B,V,B] {
def mapInner(in: V, f: (V) => B): B = f(in)
}
// drill down one level of "Map".
implicit def wrappedMapper[K,V,B,InnerV,InnerB]
(implicit innerMapper: CanMapInner[InnerV,InnerB,V,B]) =
new CanMapInner[Map[K,InnerV], Map[K,InnerB],V,B] {
def mapInner(in: Map[K, InnerV], f: (V) => B): Map[K, InnerB] =
in.mapValues(innerMapper.mapInner(_, f))
}
// the actual implementation.
def deepMapValues[K,V,B,WrappedV,WrappedB](map: Map[K,WrappedV], f: V => B)
(implicit mapper: CanMapInner[WrappedV,WrappedB,V,B]) = {
map.mapValues(inner => mapper.mapInner(inner, f))
}
// testing with a simple map
{
val initMap = Map(1 -> "Hello", 2 -> "Goodbye")
val newMap = deepMapValues(initMap, (s: String) => s.length)
println(newMap) // Map(1 -> 5, 2 -> 7)
}
// testing with a nested map
{
val initMap = Map(1 -> Map("Hi" -> "Hello"), 2 -> Map("Bye" -> "Goodbye"))
val newMap = deepMapValues(initMap, (s: String) => s.length)
println(newMap) // Map(1 -> Map(Hi -> 5), 2 -> Map(Bye -> 7))
}
Of course, in real code the pattern-matching dynamic solution is awfully tempting thanks to its simplicity. Type-safety isn't everything :)
I'm sure there is a better way using Manifest, but pattern matching seems to distinguish Seq and Map, so here it is:
object Foo {
def mapValues[A <: Map[_, _], C, D](map: A)(f: C => D): Map[_, _] = map.mapValues {
case seq: Seq[C] => seq.groupBy(f)
case innerMap: Map[_, _] => mapValues(innerMap)(f)
}
}
scala> val group0 = List("fooo", "bar", "foo") groupBy (_(0))
group0: scala.collection.immutable.Map[Char,List[java.lang.String]] = Map((f,List(fooo, foo)), (b,List(bar)))
scala> val group1 = Foo.mapValues(group0)((x: String) => x(1))
group1: scala.collection.immutable.Map[_, Any] = Map((f,Map(o -> List(fooo, foo))), (b,Map(a -> List(bar))))
scala> val group2 = Foo.mapValues(group1)((x: String) => x(2))
group2: scala.collection.immutable.Map[_, Any] = Map((f,Map(o -> Map(o -> List(fooo, foo)))), (b,Map(a -> Map(r -> List(bar)))))
Edit:
Here's a typed version using higher-kinded type.
trait NestedMapValue[Z] {
type Next[X] <: NestedMapValue[Z]
def nextValues[D](f: Z => D): Next[D]
}
trait NestedMap[Z, A, B <: NestedMapValue[Z]] extends NestedMapValue[Z] { self =>
type Next[D] = NestedMap[Z, A, B#Next[D]]
val map: Map[A, B]
def nextValues[D](f: Z => D): Next[D] = self.mapValues(f)
def mapValues[D](f: Z => D): NestedMap[Z, A, B#Next[D]] = new NestedMap[Z, A, B#Next[D]] { val map = self.map.mapValues {
case x: B => x.nextValues[D](f)
}}
override def toString = "NestedMap(%s)" format (map.toString)
}
trait Bottom[A] extends NestedMapValue[A] {
type Next[D] = NestedMap[A, D, Bottom[A]]
val seq: Seq[A]
def nextValues[D](f: A => D): Next[D] = seq match {
case seq: Seq[A] => groupBy[D](f)
}
def groupBy[D](f: A => D): Next[D] = seq match {
case seq: Seq[A] =>
new NestedMap[A, D, Bottom[A]] { val map = seq.groupBy(f).map { case (key, value) => (key, new Bottom[A] { val seq = value })} }
}
override def toString = "Bottom(%s)" format (seq.toString)
}
object Bottom {
def apply[A](aSeq: Seq[A]) = new Bottom[A] { val seq = aSeq }
}
scala> val group0 = Bottom(List("fooo", "bar", "foo")).groupBy(x => x(0))
group0: NestedMap[java.lang.String,Char,Bottom[java.lang.String]] = NestedMap(Map(f -> Bottom(List(fooo, foo)), b -> Bottom(List(bar))))
scala> val group1 = group0.mapValues(x => x(1))
group1: NestedMap[java.lang.String,Char,Bottom[java.lang.String]#Next[Char]] = NestedMap(Map(f -> NestedMap(Map(o -> Bottom(List(fooo, foo)))), b -> NestedMap(Map(a -> Bottom(List(bar))))))
scala> val group2 = group1.mapValues(x => x.size)
group2: NestedMap[java.lang.String,Char,Bottom[java.lang.String]#Next[Char]#Next[Int]] = NestedMap(Map(f -> NestedMap(Map(o -> NestedMap(Map(4 -> Bottom(List(fooo)), 3 -> Bottom(List(foo)))))), b -> NestedMap(Map(a -> NestedMap(Map(3 -> Bottom(List(bar))))))))