Reduce sequence by parts - scala

I have a sequence Seq[T] and I want to do partial reduce. For example for a Seq[Int] I want to get Seq[Int] consisting of the longest partial sums of monotonic regions. For example:
val s = Seq(1, 2, 4, 3, 2, -1, 0, 6, 8)
groupMonotionic(s) = Seq(1 + 2 + 4, 3 + 2 + (-1), 0 + 6 + 8)
I was looking for some method like conditional fold with the signature fold(z: B)((B, T) => B, (T, T) => Boolean) where the predicate states for where to terminate current sum aggregation, but it seems there is no something like that in the subtrait hierarchy of Seq.
What would be a solution using Scala Collection API and without using mutable variables?

Here is one way amongst many to do this (using Scala 2.13's List#unfold):
// val items = Seq(1, 2, 4, 3, 2, -1, 0, 6, 8)
items match {
case first :: _ :: _ => // If there are more than 2 items
List
.unfold(items.sliding(2).toList) { // We slid items to work on pairs of consecutive items
case Nil => // No more items to unfold
None // None signifies the end of the unfold
case rest # Seq(a, b) :: _ => // We span based on the sign of a-b
Some(rest.span(x => (x.head - x.last).signum == (a-b).signum))
}
.map(_.map(_.last)) // back from slided pairs
match { case head :: rest => (first :: head) :: rest }
case _ => // If there is 0 or 1 item
items.map(List(_))
}
// List(List(1, 2, 4), List(3, 2, -1), List(0, 6, 8))
List.unfold iterates as long as the unfolding function provides Some. It starts with an initial state which is the list of items to unfold. At each iteration, we span the state (remaining elements to unfold) based on the sign of the heading two elements difference. The unfolded elements are heading items sharing the same monotony and the unfolding state becomes the other remaining elements.
List#span splits a list into a tuple whose first part contains elements matching the predicate applied until the predicate stops being valid. The second part of the tuple contains the rest of the elements. Which fits perfectly the expected return type of List.unfold's unfolding function, which is Option[(A, S)] (In this case Option[(List[Int], List[Int])]).
Int.signum returns -1, 0 or 1 depending on the sign of the integer it's applied on.
Note that the first item has to be put back in the result as it hasn't an ancestor determining its signum (match { case head :: rest => (first :: head) :: rest }).
To apply the reducing function (in this case a sum), we can map the final result: .map(_.sum)

Works in Scala 2.13+ with cats
import scala.util.chaining._
import cats.data._
import cats.implicits._
val s = List(1, 2, 4, 3, 2, -1, 0, 6, 8)
def isLocalExtrema(a: List[Int]) =
a.max == a(1) || a.min == a(1)
implicit class ListOps[T](ls: List[T]) {
def multiSpanUntil(f: T => Boolean): List[List[T]] = ls.span(f) match {
case (h, Nil) => List(h)
case (h, t) => (h ::: t.take(1)) :: t.tail.multiSpanUntil(f)
}
}
def groupMonotionic(groups: List[Int]) = groups match {
case Nil => Nil
case x if x.length < 3 => List(groups.sum)
case _ =>
groups
.sliding(3).toList
.map(isLocalExtrema)
.pipe(false :: _ ::: List(false))
.zip(groups)
.multiSpanUntil(!_._1)
.pipe(Nested.apply)
.map(_._2)
.value
.map(_.sum)
}
println(groupMonotionic(s))
//List(7, 4, 14)

Here's one way using foldLeft to traverse the numeric list with a Tuple3 accumulator (listOfLists, prevElem, prevTrend) that stores the previous element and previous trend to conditionally assemble a list of lists in the current iteration:
val list = List(1, 2, 4, 3, 2, -1, 0, 6, 8)
val isUpward = (a: Int, b: Int) => a < b
val initTrend = isUpward(list.head, list.tail.head)
val monotonicLists = list.foldLeft( (List[List[Int]](), list.head, initTrend) ){
case ((lol, prev, prevTrend), curr) =>
val currTrend = isUpward(curr, prev)
if (currTrend == prevTrend)
((curr :: lol.head) :: lol.tail , curr, currTrend)
else
(List(curr) :: lol , curr, currTrend)
}._1.reverse.map(_.reverse)
// monotonicLists: List[List[Int]] = List(List(1, 2, 4), List(3, 2, -1), List(0, 6, 8))
To sum the individual nested lists:
monotonicLists.map(_.sum)
// res1: List[Int] = List(7, 4, 14)

Related

Splitting List into List of List

How can I convert:
List(1,1,1,1,4,2,2,2,2)
into:
List(List(1,1,1,1), List(2,2,2,2))
Thought this would be the easiest way to show what I'm looking for. I am having a hard time trying to find the most functional way to do this with a large list that needs to be separated at a specific element. This element does not show up in the new list of lists. Any help would be appreciated!
If you want to support multiple instances of that separator, you can use foldRight with some list "gymnastics":
// more complex example: separator (4) appears multiple times
val l = List(1,1,1,1,4,2,2,2,2,4,5,6,4)
val separator = 4
val result = l.foldRight(List[List[Int]]()) {
case (`separator`, res) => List(Nil) ++ res
case (v, head :: tail) => List(v :: head) ++ tail
case (v, Nil) => List(List(v))
}
// result: List(List(1, 1, 1, 1), List(2, 2, 2, 2), List(5, 6))
This is the cleanest way to do this
val (l, _ :: r) = list.span( _ != 4)
The span function splits the list at the first value not matching the condition, and the de-structuring on the left-hand side removes the matching value from the second list.
This will fail if there is no matching value.
Given a list and a delimiter, in order to split the list in 2:
val list = List(1, 1, 1, 1, 4, 2, 2, 2, 2)
val delimiter = 4
you could use a combination of List.indexOf, List.take and List.drop:
val splitIdx = list.indexOf(delimiter)
List(list.take(splitIdx), list.drop(splitIdx + 1))
you could use List.span which splits the list into a tuple given a predicate:
list.span(_ != delimiter) match { case (l1, l2) => List(l1, l2.tail) }
in order to produce:
List(List(1, 1, 1, 1), List(2, 2, 2, 2))
scala> val l = List(1,1,1,1,4,2,2,2,2)
l: List[Int] = List(1, 1, 1, 1, 4, 2, 2, 2, 2)
scala> l.splitAt(l.indexOf(4))
res0: (List[Int], List[Int]) = (List(1, 1, 1, 1),List(4, 2, 2, 2, 2))
def convert(list: List[Int], separator: Int): List[List[Int]] = {
#scala.annotation.tailrec
def rec(acc: List[List[Int]], listTemp: List[Int]): List[List[Int]] = {
if (listTemp.isEmpty) acc
else {
val (l, _ :: r) = listTemp.span(_ != separator)
rec(acc ++ List(l), r)
}
}
rec(List(), list)
}

Scala - map function to replace negatives with previous number in list

I have a list of numbers
[1, 2, 3, -1000, 4, -1000]
I want to write a map function to replace all negative numbers in my list with the previous number (before the negative)
In this case the output will be
[1, 2, 3, 3, 4, 4]
What is the best way to write this map function?
yourList.foldLeft(List[Int]()) { (acc, i) => if (i >= 0) i :: acc else acc.head :: acc }.reverse
will throw an exception if first number is negative.
With thanks to Aivean.
def replaceNegatives(list: List[Int], prev: Int = 0): List[Int] = {
list match {
case Nil => Nil
case (x :: xs) if x < 0 => prev :: replaceNegatives(xs, prev)
case x :: xs => x :: replaceNegatives(xs, x)
}
}
Example:
scala> replaceNegatives(List(1, 2, 3, -1000, 4, -1000))
res1: List[Int] = List(1, 2, 3, 3, 4, 4)
The second argument (prev) is optional; it is the default value to use if the first item in the list is negative.

Collapse subsequence of matching elements in a Scala sequence

Given a Scala sequence...
val sequence: Seq = List( 3, 1, 4, 1, 5, 9, 2, 6, 5 )
...say that I want to find all subsequences matching certain criteria, e.g. strings of odd numbers, and replace those with the result of some operation on that subsequence, say its length, producing a new sequence:
val sequence2: Seq = List( 2, 4, 3, 2, 6, 1 )
(Yes, this is a fairly contrived example, but it's concise.)
So far the best I've been able to do is this ugly imperative hack:
val sequence: Seq[Int] = List( 3, 1, 4, 1, 5, 9, 2, 6, 5 )
var sequence2 = List[Int]() // this is going to be our result
var subsequence = List[Int]()
for (s <- sequence) {
if (s % 2 == 0) {
if (!subsequence.isEmpty) {
sequence2 = sequence2 :+ subsequence.length
subsequence = List[Int]()
}
sequence2 = sequence2 :+ s
} else {
subsequence = subsequence :+ s
}
}
if (!subsequence.isEmpty) {
sequence2 = sequence2 :+ subsequence.length
}
Is there an elegant (/ functional) way to do this?
Using multiSpan for spanning a list into sublists by given criteria, consider this solution for the problem depicted above,
sequence.multiSpan( _ % 2 == 0 ).flatMap {
case h :: xs if h % 2 != 0 => List( (h::xs).length)
case h :: Nil => List(h)
case h :: xs => List(h, xs.length) }
Note that
sequence.multiSpan( _ % 2 == 0 )
List(List(3, 1), List(4, 1, 5, 9), List(2), List(6, 5))
and hence we flatMap these nested lists by considering three cases: whether the condition does not hold and so we apply a function; whether it is a singleton list (the condition holds); or else whether the first element holds and the rest needs be applied a function.
What you're looking for is a fold:
sequence.foldLeft(List(0)) { (soFar, next) =>
if(next % 2 == 0) soFar :+ next :+ 0 else soFar.init :+ (soFar.last + 1)
}.filter(_ != 0)
Or in a different style:
(List(0) /: sequence) {
case(soFar, next) if next % 2 == 0 => soFar :+ next :+ 0
case(soFar, _) => soFar.init :+ (soFar.last + 1)
}.filter(_ != 0)
Or with a foldRight instead, which is sometimes more performant:
(sequence :\ List(0)) {
case(next, soFar) if next % 2 == 0 => 0 :: next :: soFar
case(_, hd::tl) => (hd + 1)::tl
}.filter(_ != 0).reverse
You can read more about fold, foldLeft, foldRight, and other useful functions here and here.
I originally thought you were asking for all subsequences of a sequence. That might be useful in similar situations, so I'll leave this here. You can use inits and tails together to get all subsequences, and then use filter and map for your purposes:
val sequence = List( 3, 1, 4, 1, 5, 9, 2, 6, 5 )
val subsequences = sequence.tails.flatMap(_.inits).toList.distinct
subsequences.filter(_.forall(_ % 2 == 1)).map(_.length)
Here is my attempt at a recursive implementation
def subSequenceApply(list: List[Int], predicate: (Int)=> Boolean, func: (List[Int]) => Int):List[Int] = list match {
case Nil => Nil
case h :: t if !predicate(h) => h :: subSequenceApply(t, predicate, func)
case _ =>
val (matchSeq,nonMatch) = list.span(predicate)
func(matchSeq) :: subSequenceApply(nonMatch, predicate, func)
}
So given sequence in your example. You could run it as
subSequenceApply(sequence, _ % 2 != 0, _.length)

How to return all positives and the first negative number in a list using functional programming?

Imagine I have an unsorted list of positive & negative ints. I want to return a list containing all the positive ints, and the first negative number, but then ignore all subsequent negative numbers from the list, while preserving the ordering.
Imperatively I could do:
l = [1, 2, -4, 5, -6, -1, 3]
out = []
first = true
for n in l:
if n >= 0:
out.push(n)
else if first:
out.push(n)
first = false
// out = [1, 2, -4, 5, 3]
How could I do this with FP in Scala? I was thinking (probably won't compile...):
val l = List(1, 2, -4, 5, -6, -1, 3)
val posl = l.map(_ >= 0)
val negl = l.zipWithIndex.map((n, i) => if (n < 0) (i, n) else (None, None)).head
// now split posl at negl._1, and create a new list of leftSlice :: negl._2 :: rightSlice?
Is this the right approach, or is there a more elegant, succinct way?
It wouldn't be a proper functional programming question without a slightly-too-clever recursion+pattern matching answer.
def firstNegAllPos(l:List[Int]):List[Int] = {
l match{
case x::xs if x>=0 => x::firstNegAllPos(xs)
case x::xs if x<0 => x::xs.filter(_>=0)
case Nil => Nil
}
}
Here is a tail-recursive way. Compared to m-z's answer, it iterates your list only one time, compared to Dimas answer, it does not use mutable state, so it is pure functional.
def firstNegAllPos(list: List[Int]) : List[Int] = {
def loop(in: List[Int], out: List[Int], negfound: Boolean) : List [Int] = {
in match {
case Nil => out
case head :: tail =>
if (negfound)
loop(tail, if (head < 0) out else head :: out, true)
else
loop(tail, head :: out, head < 0)
}
}
loop(list, Nil, false)
}
firstNegAllPos(List(1, 2, -4, 5, -6, -1, 3)) // List(3, 5, -4, 2, 1)
Edit:
The above implementation provides a reversed result. In order to preserve order you can do following:
def firstNegAllPos(list: List[Int]) : List[Int] = {
def loop(in: List[Int], out: List[Int], negfound: Boolean) : List [Int] = {
in match {
case Nil => out
case head :: tail =>
if (negfound)
loop(tail, if (head < 0) out else head :: out, true)
else
loop(tail, head :: out, head < 0)
}
}
loop(list, Nil, false).reverse
}
firstNegAllPos(List(1, 2, -4, 5, -6, -1, 3)) // List(1, 2, -4, 5, 3)
You can do it in one pass provided you don't mind keeping a bit of state around - mind the var neg:
var neg = false
list.filter {
case x if x > 0 => true
case _ if !neg => neg = true
}
A direct translation of the requirement is pretty clear, takes one pass over the list, and is functional:
val l = List(1, 2, -4, 5, -6, -1, 3)
// split into any initial positive numbers, and the rest of the list
val (pos, firstneg) = l.span(_ >= 0)
// if there were no negative numbers, return the original list.
// otherwise, it's the initial positives, the first negative, and
// the positive numbers from the rest of the list.
if (firstNeg.isEmpty) l else pos:::List(firstneg.head):::firstneg.tail.filter(_>=0)
//> res0: List[Int] = List(1, 2, -4, 5, 3)
(The List around firstneg.head is just for the symmetry of ::: both sides)
val l = List(1, 2, -4, 5, -6, -1, 3)
val (left, right) = l.span(_ > 0)
val result = right.headOption match {
case Some(n) => (left :+ n) ++ right.tail.filter(_ > 0)
case None => left
}
This is an obvious work for fold operation!
val l = List(1, 2, -4, 5, -6, -1, 3)
var result = l.foldLeft((true, Vector.empty[Int])) {
case ((f, r), x) if x >= 0 => (f, r :+ x)
case ((f, r), x) if f => (false, r :+ x)
case ((f, r), x) => (f, r)
}._2
println(result) // Vector(1, 2, -4, 5, 3)
I used a vector as an intermediate structure; you can convert it to a list with toList, if you need it. Or you can use it instead of the Vector, but you will have to reverse the addition order (r :+ x => x :: r) and then reverse the list in the end with reverse method.
If you want to maintain the order (i.e., the position of the negative), you could do something like this:
val list = List(1, 2, -4, 5, -6, -1, 3)
val negIndex = list.indexWhere(_ < 0)
val positive = list.zipWithIndex.filter { case (num, index) =>
num >= 0 || index == negIndex
}.map(_._1)
The negative requirement makes it hard to keep it more succinct than that. My strategy is to just grab the index of the first negative with indexWhere (will be -1 if there is none), then filter all the negatives out of the list, except for the index of the first negative. If the index was -1, no harm, no foul.
Here is tail recursive solution preserving order:
def posNeg(xs: List[Int]): List[Int] = {
#tailrec
def go(tail: List[Int], res: List[Int]): List[Int] = {
tail match {
case h :: t =>
if (h >= 0) go(t, h :: res)
else (h :: res).reverse ::: t.filter(_ > 0)
case _ => res
}
}
go(xs, Nil)
}
There are a lot of tail-recursive solutions, but they're all longer than they need to be.
def oneNeg(xs: List[Int]): List[Int] = {
def loop(in: List[Int], out: List[Int], neg: Int): List[Int] = in match {
case Nil => out
case x :: rest =>
if (neg < 0 && x < 0) loop(rest, out, neg)
else loop(rest, x :: out, x min neg)
}
loop(xs, Nil, 0).reverse
}
If this is not a public-facing API, you can make it shorter yet by just exposing the inner method alone:
def oneNeg(in: List[Int], out: List[Int] = Nil, neg: Int = 0): List[Int] =
in match {
case Nil => out.reverse
case x :: rest =>
if (neg < 0 && x < 0) oneNeg(rest, out, neg)
else oneNeg(rest, x :: out, x min neg)
}
Here is an improvement on the most concise answer I saw, by #sschaef in a comment:
val l = List(1, 2, -4, 5, -6, -1, 3)
l.span(_>=0) match { case (left, Nil) => left
case (left, x::right) => left ++ (x +: right.filter(_>=0)) }
You can try:
val list = List(1, 2, -4, 5, -6, -1, 3)
val index = list.indexWhere(_ < 0) + 1
list.take(index) ++ list.drop(index).filter(_ > 0)

How to remove 2 or more duplicates from list and maintain their initial order?

Lets assume we have a Scala list:
val l1 = List(1, 2, 3, 1, 1, 3, 2, 5, 1)
We can easily remove duplicates using the following code:
l1.distinct
or
l1.toSet.toList
But what if we want to remove duplicates only if there are more than 2 of them? So if there are more than 2 elements with the same value we remain only two and remove the rest of them.
I could achieve it with following code:
l1.groupBy(identity).mapValues(_.take(2)).values.toList.flatten
that gave me the result:
List(2, 2, 5, 1, 1, 3, 3)
Elements are removed but the order of remaining elements is different from how these elements appeared in the initial list. How to do this operation and remain the order from original list?
So the result for l1 should be:
List(1, 2, 3, 1, 3, 2, 5)
Not the most efficient.
scala> val l1 = List(1, 2, 3, 1, 1, 3, 2, 5, 1)
l1: List[Int] = List(1, 2, 3, 1, 1, 3, 2, 5, 1)
scala> l1.zipWithIndex.groupBy( _._1 ).map(_._2.take(2)).flatten.toList.sortBy(_._2).unzip._1
res10: List[Int] = List(1, 2, 3, 1, 3, 2, 5)
My humble answer:
def distinctOrder[A](x:List[A]):List[A] = {
#scala.annotation.tailrec
def distinctOrderRec(list: List[A], covered: List[A]): List[A] = {
(list, covered) match {
case (Nil, _) => covered.reverse
case (lst, c) if c.count(_ == lst.head) >= 2 => distinctOrderRec(list.tail, covered)
case _ => distinctOrderRec(list.tail, list.head :: covered)
}
}
distinctOrderRec(x, Nil)
}
With the results:
scala> val l1 = List(1, 2, 3, 1, 1, 3, 2, 5, 1)
l1: List[Int] = List(1, 2, 3, 1, 1, 3, 2, 5, 1)
scala> distinctOrder(l1)
res1: List[Int] = List(1, 2, 3, 1, 3, 2, 5)
On Edit: Right before I went to bed I came up with this!
l1.foldLeft(List[Int]())((total, next) => if (total.count(_ == next) >= 2) total else total :+ next)
With an answer of:
res9: List[Int] = List(1, 2, 3, 1, 3, 2, 5)
Not the prettiest. I look forward to seeing the other solutions.
def noMoreThan(xs: List[Int], max: Int) =
{
def op(m: Map[Int, Int], a: Int) = {
m updated (a, m(a) + 1)
}
xs.scanLeft( Map[Int,Int]().withDefaultValue(0) ) (op).tail
.zip(xs)
.filter{ case (m, a) => m(a) <= max }
.map(_._2)
}
scala> noMoreThan(l1, 2)
res0: List[Int] = List(1, 2, 3, 1, 3, 2, 5)
More straightforward version using foldLeft:
l1.foldLeft(List[Int]()){(acc, el) =>
if (acc.count(_ == el) >= 2) acc else el::acc}.reverse
Similar to how distinct is implemeted, with a multiset instead of a set:
def noMoreThan[T](list : List[T], max : Int) = {
val b = List.newBuilder[T]
val seen = collection.mutable.Map[T,Int]().withDefaultValue(0)
for (x <- list) {
if (seen(x) < max) {
b += x
seen(x) += 1
}
}
b.result()
}
Based on experquisite's answer, but using foldLeft:
def noMoreThanBis(xs: List[Int], max: Int) = {
val initialState: (Map[Int, Int], List[Int]) = (Map().withDefaultValue(0), Nil)
val (_, result) = xs.foldLeft(initialState) { case ((count, res), x) =>
if (count(x) >= max)
(count, res)
else
(count.updated(x, count(x) + 1), x :: res)
}
result.reverse
}
distinct is defined for SeqLike as
/** Builds a new $coll from this $coll without any duplicate elements.
* $willNotTerminateInf
*
* #return A new $coll which contains the first occurrence of every element of this $coll.
*/
def distinct: Repr = {
val b = newBuilder
val seen = mutable.HashSet[A]()
for (x <- this) {
if (!seen(x)) {
b += x
seen += x
}
}
b.result()
}
We can define our function in very similar fashion:
def distinct2[A](ls: List[A]): List[A] = {
val b = List.newBuilder[A]
val seen1 = mutable.HashSet[A]()
val seen2 = mutable.HashSet[A]()
for (x <- ls) {
if (!seen2(x)) {
b += x
if (!seen1(x)) {
seen1 += x
} else {
seen2 += x
}
}
}
b.result()
}
scala> distinct2(l1)
res4: List[Int] = List(1, 2, 3, 1, 3, 2, 5)
This version uses internal state, but is still pure. It is also quite easy to generalise for arbitrary n (currently 2), but specific version is more performant.
You can implement the same function with folds carrying the "what is seen once and twice" state with you. Yet the for loop and mutable state does the same job.
How about this:
list
.zipWithIndex
.groupBy(_._1)
.toSeq
.flatMap { _._2.take(2) }
.sortBy(_._2)
.map(_._1)
Its a bit ugly, but its relatively faster
val l1 = List(1, 2, 3, 1, 1, 3, 2, 5, 1)
l1.foldLeft((Map[Int, Int](), List[Int]())) { case ((m, ls), x) => {
val z = m + ((x, m.getOrElse(x, 0) + 1))
(z, if (z(x) <= 2) x :: ls else ls)
}}._2.reverse
Gives: List(1, 2, 3, 1, 3, 2, 5)
Here is a recursive solution (it will stack overflow for large lists):
def filterAfter[T](l: List[T], max: Int): List[T] = {
require(max > 1)
//keep the state of seen values
val seen = Map[T, Int]().withDefaultValue(0)//init to 0
def filterAfter(l: List[T], seen: Map[T, Int]): (List[T], Map[T, Int]) = {
l match {
case x :: xs =>
if (seen(x) < max) {
//Update the state and pass to next
val pair = filterAfter(xs, seen updated (x, seen(x) + 1))
(x::pair._1, pair._2)
} else {
//already seen more than max
filterAfter(xs, seen)
}
case _ => (l, seen)//empty, terminate recursion
}
}
//call inner recursive function
filterAfter(l, seen, 2)._1
}
Here is canonical Scala code to do reduce three or more in a row to two in a row:
def checkForTwo(candidate: List[Int]): List[Int] = {
candidate match {
case x :: y :: z :: tail if x == y && y == z =>
checkForTwo(y :: z :: tail)
case x :: tail =>
x :: checkForTwo(tail)
case Nil =>
Nil
}
}
It looks at the first three elements of the list, and if they are the same, drops the first one and repeats the process. Otherwise, it passes items on through.
Solution with groupBy and filter, without any sorting (so it's O(N), sorting will give you additional O(Nlog(N)) in typical case):
val li = l1.zipWithIndex
val pred = li.groupBy(_._1).flatMap(_._2.lift(1)) //1 is your "2", but - 1
for ((x, i) <- li if !pred.get(x).exists(_ < i)) yield x
I prefer approach with immutable Map:
def noMoreThan[T](list: List[T], max: Int): List[T] = {
def go(tail: List[T], freq: Map[T, Int]): List[T] = {
tail match {
case h :: t =>
if (freq(h) < max)
h :: go(t, freq + (h -> (freq(h) + 1)))
else go(t, freq)
case _ => Nil
}
}
go(list, Map[T, Int]().withDefaultValue(0))
}