Abstract over repeated flatMap - scala

I am trying to generalize repeated, nested flatMap but not sure if one exists.
The following code will produce all combinations of n choose 3, :
def choose3flatMap(n: Int, r: Int = 3) =
(0 to n - r)
.flatMap(i => (i + 1 to n - (r - 1))
.flatMap(j => (j + 1 to n - (r - 2))
.map(k => Seq(i, j, k))))
Repeating the flatMap operation, we can get all combinations of n choose 5, :
def choose5flatMap(n: Int, r: Int = 5) =
(0 to n - r)
.flatMap(i => (i + 1 to n - (r - 1))
.flatMap(j => (j + 1 to n - (r - 2))
.flatMap(k => (k + 1 to n - (r - 3))
.flatMap(l => (l + 1 to n - (r - 4))
.map(m => Seq(i, j, k, l, m)))))
Clearly there is a pattern here. I would like to utilize this similarity to get a general solution for n choose r, . Is there a simple way to accomplish this. Perhaps a higher order function of some sort?
What I have tried:
Scala lets me rewrite the map/flatMap with a for expression. This reads cleaner, but the number of choices in still hard-coded.
def choose3Loop(n: Int, r: Int = 3) =
for {
i <- 0 to n - r
j <- i + 1 to n - (r - 1)
k <- j + 1 to n - (r - 2)
} yield Seq(i, j, k)
I can write a recursive solution directly using flatMap or utilizing the sugar of a for expression:
def combinationsRecursive(n: Int, r: Int, i: Int = 0): Seq[Seq[Int]] =
if (r == 1) (i until n).map(Seq(_))
else {
(i to n - r).flatMap(
i => combinationsRecursive(n, r - 1, i + 1).map(j => i +: j))
}
def combinationsRecursiveLoop(n: Int, r: Int, i: Int = 0): Seq[Seq[Int]] =
if (r == 1) (i until n).map(Seq(_))
else
for {
i <- i to n - r
j <- combinationsRecursiveLoop(n, r - 1, i + 1)
} yield i +: j
While these are solutions to the general problem, I wonder if there is a higher-level abstraction I am missing here that may be applicable to other problems as well. I recognize that for this particular application, I could do (0 to n).combinations(r) to use a library-provided implementation of computing combinations.
While the above code is Scala, in this case I am interested the functional programming aspect of it and not the language capabilities. If there is a solution but one that is not supported by Scala I am interested in that.
Edit: He is a sample caller and the resulting output by request:
scala> combinationsRecursiveLoop(5, 3)
res0: Seq[Seq[Int]] = Vector(List(0, 1, 2), List(0, 1, 3), List(0, 1, 4), List(0, 2, 3), List(0, 2, 4), List(0, 3, 4), List(1, 2, 3), List(1, 2, 4), List(1, 3, 4), List(2, 3, 4))
scala> combinationsRecursiveLoop(5, 3).map("("+_.mkString(", ")+")").mkString(" ")
res1: String = (0, 1, 2) (0, 1, 3) (0, 1, 4) (0, 2, 3) (0, 2, 4) (0, 3, 4) (1, 2, 3) (1, 2, 4) (1, 3, 4) (2, 3, 4)
It just provides all r-element subsets of the set of integers starting at zero containing n elements. More information on combinations can be found on Wikipedia.

Here is one way to look at this, that I have come up with.
You can extract one stage in your chain as a function f: List[Int] => List[List[Int]], that takes a List with a beginning of a combination, and prepends all possible next elements to it.
For example in choose(5, 3), f(List(2, 0)) would result in List(List(3, 2, 0), List(4, 2, 0)).
Here is a possible implementation of such a function with some processing for the initial case added:
val f: List[Int] => List[List[Int]] = l =>
(l.headOption.map(_ + 1).getOrElse(0) to n - (r - l.size))
.map(_ :: l).toList
Now, such a function is a Kleisli arrow Kleisli[List, List[Int], List[Int]], and it's endomorphic (has the same argument and return types).
There is a monoid instance for endomorphic kleisli arrows, where the monoid "addition" means the flatMap operation (or in pseudocode, f1 |+| f2 == a => f1(a).flatMap(f2)). So to replace your chain of flatMaps you need to "add" r instances of this f function, or in other words to multiply the f function by r.
This idea translates directly into Scalaz code:
import scalaz._, Scalaz._
def choose(n: Int, r: Int) = {
val f: List[Int] => List[List[Int]] = l =>
(l.headOption.map(_ + 1).getOrElse(0) to n - (r - l.size))
.map(_ :: l).toList
Endomorphic.endoKleisli(f).multiply(r).run(Nil)
}
And here is an example running it:
scala> choose(4, 3)
res1: List[List[Int]] = List(List(2, 1, 0), List(3, 1, 0), List(3, 2, 0), List(3, 2, 1))
The combinations are reversed, but it should be possible to make a version, that produces combinations with elements in the increasing order (or just run choose(n, r).map(_.reverse)).
Another improvement would be to make a lazy version, that returns Stream[List[Int]] (or even better a scalaz.EphemeralStream[List[Int]]: you don't want to have all the combinations cached in memory), but this is left as an exercise to the reader.

Related

Swap all Integers of a List with Scala

If I have a swap method, which can swap two Integers in a List:
def swap[E]: (List[E], Int, Int) => List[E] = (ls, i, j) =>
ls.updated(i, ls(j)).updated(j, ls(i))
And now I want to swap all Integers of a list using this method. That means the result should be like this:
swapAll(List(1,2,3)) == List(List(2,1,3), List(3,2,1), List(1,3,2))
I thought of something like this:
def swapAll: List[Int] => List[List[Int]] = ls => for(i <- 0 to ls.length; j <- i to ls.length) yield List(swap(ps, i, j))
But it doesn't work, does somebody have an Idea?
Almost there.
def swap[E](ls: List[E], i: Int, j: Int): List[E] = {
ls.updated(i, ls(j)).updated(j, ls(i))
}
def swapAll(ps: List[Int]): List[List[Int]] = {
val n = ps.size
(for {
i <- 0 until n
j <- (i + 1) until n
} yield swap(ps, i, j))(collection.breakOut)
}
Example:
swapAll(List(1, 2, 3))
// List(List(2, 1, 3), List(3, 2, 1), List(1, 3, 2))
The breakOut is a special explicitly inserted CanBuildFrom. It's necessary because without it, the type of the produced collection is some weird IndexedSequence derived from the 0 until n Range, but you want a List instead.

Looking for an FP ranking implementation which handles ties (i.e. equal values)

Starting from a sorted sequence of values, my goal is to assign a rank to each value, using identical ranks for equal values (aka ties):
Input: Vector(1, 1, 3, 3, 3, 5, 6)
Output: Vector((0,1), (0,1), (1,3), (1,3), (1,3), (2,5), (3,6))
A few type aliases for readability:
type Rank = Int
type Value = Int
type RankValuePair = (Rank, Value)
An imperative implementation using a mutable rank variable could look like this:
var rank = 0
val ranked1: Vector[RankValuePair] = for ((value, index) <- values.zipWithIndex) yield {
if ((index > 0) && (values(index - 1) != value)) rank += 1
(rank, value)
}
// ranked1: Vector((0,1), (0,1), (1,3), (1,3), (1,3), (2,5), (3,6))
To hone my FP skills, I was trying to come up with a functional implementation:
val ranked2: Vector[RankValuePair] = values.sliding(2).foldLeft((0 , Vector.empty[RankValuePair])) {
case ((rank: Rank, rankedValues: Vector[RankValuePair]), Vector(currentValue, nextValue)) =>
val newRank = if (nextValue > currentValue) rank + 1 else rank
val newRankedValues = rankedValues :+ (rank, currentValue)
(newRank, newRankedValues)
}._2
// ranked2: Vector((0,1), (0,1), (1,3), (1,3), (1,3), (2,5))
It is less readable, and – more importantly – is missing the last value (due to using sliding(2) on an odd number of values).
How could this be fixed and improved?
This works well for me:
// scala
val vs = Vector(1, 1, 3, 3, 3, 5, 6)
val rank = vs.distinct.zipWithIndex.toMap
val result = vs.map(i => (rank(i), i))
The same in Java 8 using Javaslang:
// java(slang)
Vector<Integer> vs = Vector(1, 1, 3, 3, 3, 5, 6);
Function<Integer, Integer> rank = vs.distinct().zipWithIndex().toMap(t -> t);
Vector<Tuple2<Integer, Integer>> result = vs.map(i -> Tuple(rank.apply(i), i));
The output of both variants is
Vector((0, 1), (0, 1), (1, 3), (1, 3), (1, 3), (2, 5), (3, 6))
*) Disclosure: I'm the creator of Javaslang
This is nice and concise but it assumes that your Values don't go negative. (Actually it just assumes that they can never start with -1.)
val vs: Vector[Value] = Vector(1, 1, 3, 3, 3, 5, 6)
val rvps: Vector[RankValuePair] =
vs.scanLeft((-1,-1)){ case ((r,p), v) =>
if (p == v) (r, v) else (r + 1, v)
}.tail
edit
Modification that makes no assumptions, as suggested by #Kolmar.
vs.scanLeft((0,vs.headOption.getOrElse(0))){ case ((r,p), v) =>
if (p == v) (r, v) else (r + 1, v)
}.tail
Here's an approach with recursion, pattern matching and guards.
The interesting part is where the head and head of the tail (h and ht respectively) are de-constructed from the list and an if checks if they are equal. The logic for each case adjusts the rank and proceeds on the remaining part of the list.
def rank(xs: Vector[Value]): List[RankValuePair] = {
def rankR(xs: List[Value], acc: List[RankValuePair], rank: Rank): List[RankValuePair] = xs match{
case Nil => acc.reverse
case h :: Nil => rankR(Nil, (rank, h) :: acc, rank)
case h :: ht :: t if (h == ht) => rankR(xs.tail, (rank, h) :: acc, rank)
case h :: ht :: t if (h != ht) => rankR(xs.tail, (rank, h) :: acc, rank + 1)
}
rankR(xs.toList, List[RankValuePair](), 0)
}
Output:
scala> rank(xs)
res14: List[RankValuePair] = List((0,1), (0,1), (1,3), (1,3), (1,3), (2,5), (3,6))
This is a modification of the solution by #jwvh, that doesn't make any assumptions about the values:
val vs = Vector(1, 1, 3, 3, 3, 5, 6)
vs.sliding(2).scanLeft(0, vs.head) {
case ((rank, _), Seq(a, b)) => (if (a != b) rank + 1 else rank, b)
}.toVector
Note, that it would throw if vs is empty, so you'd have to use vs.headOption getOrElse 0, or check if the input is empty beforehand: if (vs.isEmpty) Vector.empty else ...
import scala.annotation.tailrec
type Rank = Int
// defined type alias Rank
type Value = Int
// defined type alias Value
type RankValuePair = (Rank, Value)
// defined type alias RankValuePair
def rankThem(values: List[Value]): List[RankValuePair] = {
// Assumes that the "values" are sorted
#tailrec
def _rankThem(currentRank: Rank, currentValue: Value, ranked: List[RankValuePair], values: List[Value]): List[RankValuePair] = values match {
case value :: tail if value == currentValue => _rankThem(currentRank, value, (currentRank, value) +: ranked, tail)
case value :: tail if value > currentValue => _rankThem(currentRank + 1, value, (currentRank + 1, value) +: ranked, tail)
case Nil => ranked.reverse
}
_rankThem(0, Int.MinValue, List.empty[RankValuePair], values.sorted)
}
// rankThem: rankThem[](val values: List[Value]) => List[RankValuePair]
val valueList = List(1, 1, 3, 3, 5, 6)
// valueList: List[Int] = List(1, 1, 3, 3, 5, 6)
val rankValueList = rankThem(valueList)[RankedValuePair], values: Vector[Value])
// rankValueList: List[RankValuePair] = List((1,1), (1,1), (2,3), (2,3), (3,5), (4,6))
val list = List(1, 1, 3, 3, 5, 6)
val result = list
.groupBy(identity)
.mapValues(_.size)
.toArray
.sortBy(_._1)
.zipWithIndex
.flatMap(tuple => List.fill(tuple._1._2)((tuple._2, tuple._1._1)))
result: Array[(Int, Int)] = Array((0,1), (0,1), (1,3), (1,3), (2,5), (3,6))
The idea is using groupBy to find identical elements and find their occurrences and then sort and then flatMap. Time complexity I would say is O(nlogn), groupBy is O(n), sort is O(nlogn), fl

my combinations function returns an empty list

I am working on S-99: Ninety-Nine Scala Problems and already stuck at question 26.
Generate the combinations of K distinct objects chosen from the N elements of a list.
After wasting a couple hours, I decided to peek at a solution written in Haskell:
combinations :: Int -> [a] -> [[a]]
combinations 0 _ = [ [] ]
combinations n xs = [ y:ys | y:xs' <- tails xs
, ys <- combinations (n-1) xs']
It looks pretty straightforward so I decided to translate into Scala. (I know that's cheating.) Here's what I got so far:
def combinations[T](n: Int, ls: List[T]): List[List[T]] = (n, ls) match {
case (0, _) => List[List[T]]()
case (n, xs) => {
for {
y :: xss <- allTails(xs).reverse
ys <- combinations((n - 1), xss)
} yield y :: ys
}
}
My helper function:
def allTails[T](ls: List[T]): List[List[T]] = {
ls./:(0, List[List[T]]())((acc, c) => {
(acc._1 + 1, ls.drop(acc._1) :: acc._2)
})._2 }
allTails(List(0, 1, 2, 3)).reverse
//> res1: List[List[Int]] = List(List(0, 1, 2, 3), List(1, 2, 3), List(2, 3), List(3))
However, my combinations returns an empty list. Any idea?
Other solutions with explanation are very welcome as well. Thanks
Edit: The description of the question
Generate the combinations of K distinct objects chosen from the N elements of a list.
In how many ways can a committee of 3 be chosen from a group of 12 people? We all know that there are C(12,3) = 220 possibilities (C(N,K) denotes the well-known binomial coefficient). For pure mathematicians, this result may be great. But we want to really generate all the possibilities.
Example:
scala> combinations(3, List('a, 'b, 'c, 'd, 'e, 'f))
res0: List[List[Symbol]] = List(List('a, 'b, 'c), List('a, 'b, 'd), List('a, 'b, 'e), ...
As Noah pointed out, my problem is for of an empty list doesn't yield. However, the hacky work around that Noah suggested is wrong. It adds an empty list to the result of every recursion step. Anyway, here is my final solution. I changed the base case to "case (1, xs)". (n matches 1)
def combinations[T](n: Int, ls: List[T]): List[List[T]] = (n, ls) match {
case (1, xs) => xs.map(List(_))
case (n, xs) => {
val tails = allTails(xs).reverse
for {
y :: xss <- allTails(xs).reverse
ys <- combinations((n - 1), xss)
} yield y :: ys
}
}
//combinations(3, List(1, 2, 3, 4))
//List(List(1, 2, 3), List(1, 2, 4), List(1, 3, 4), List(2, 3, 4))
//combinations(2, List(0, 1, 2, 3))
//List(List(0, 1), List(0, 2), List(0, 3), List(1, 2), List(1, 3), List(2, 3))
def allTails[T](ls: List[T]): List[List[T]] = {
ls./:(0, List[List[T]]())((acc, c) => {
(acc._1 + 1, ls.drop(acc._1) :: acc._2)
})._2
}
//allTails(List(0,1,2,3))
//List(List(3), List(2, 3), List(1, 2, 3), List(0, 1, 2, 3))
You made a mistake when translating the Haskell version here:
case (0, _) => List[List[T]]()
This returns an empty list. Whereas the Haskell version
combinations 0 _ = [ [] ]
returns a list with a single element, and that element is an empty list.
This is essentially saying that there is one way to choose zero items, and that is important because the code builds on this case recursively for the cases where we choose more items. If there were no ways to select zero items, then there would also be no ways to select one item and so on. That's what's happening in your code.
If you fix the Scala version to do the same as the Haskell version:
case (0, _) => List(List[T]())
it works as expected.
Your problem is using the for comprehension with lists. If the for detects an empty list, then it short circuits and returns an empty list instead of 'cons'ing your head element. Here's an example:
scala> for { xs <- List() } yield println("It worked!") // This never prints
res0: List[Unit] = List()
So, a kind of hacky work around for your combinations function would be:
def combinations[T](n: Int, ls: List[T]): List[List[T]] = (n, ls) match {
case (0, _) => List[List[T]]()
case (n, xs) => {
val tails = allTails(xs).reverse
println(tails)
for {
y :: xss <- tails
ys <- Nil :: combinations((n - 1), xss) //Now we're sure to keep evaulating even with an empty list
} yield y :: ys
}
}
scala> combinations(2, List(1, 2, 3))
List(List(1, 2, 3), List(2, 3), List(3))
List(List(2, 3), List(3))
List(List(3))
List()
res5: List[List[Int]] = List(List(1), List(1, 2), List(1, 3), List(2), List(2, 3), List(3))
One more way of solving it.
def combinations[T](n: Int, ls: List[T]): List[List[T]] = {
var ms: List[List[T]] = List[List[T]]();
val len = ls.size
if (n > len)
throw new Error();
else if (n == len)
List(ls)
else if (n == 1)
ls map (a => List(a))
else {
for (i <- n to len) {
val take: List[T] = ls take i;
val temp = combinations(n - 1, take.init) map (a => take.last :: a)
ms = ms ::: temp
}
ms
}
}
So combinations(2, List(1, 2, 3)) gives: List[List[Int]] = List(List(2, 1), List(3, 1), List(3, 2))

Stream of running sums in Scala

This is a follow-up to my previous question.
Given function add_stream(s1:Stream[Int], s2:Stream[Int]):Stream[Int]
I would like to code running_sums(s:Stream[Int]):Stream[Int], which returns a new stream : s1, s1 + s2, s1 + s2 + s3, ...
I can think of the following implementation but it does not work if s is empty
def running_sums(s:Stream[Int]):Stream[Int] =
Stream.cons(s.head, add_streams(s.tail, running_sums(s)))
I can fix it as follows:
def running_sums(s:Stream[Int]):Stream[Int] =
if (s.isEmpty) empty
else Stream.cons(s.head, add_streams(s.tail, running_sums(s)))
However it does not look elegant.
How would you implement running_sums?
There's a library call for something like this, called scanLeft
s.scanLeft(0)(_+_).tail
What about scanLeft?
scala> val sums = stream.scanLeft(List(0))((ns, n) => ns :+ (ns.last + n))
sums: scala.collection.immutable.Stream[List[Int]] = Stream(List(0), ?)
scala> sums take 5 foreach println
List(0)
List(0, 1)
List(0, 1, 3)
List(0, 1, 3, 6)
List(0, 1, 3, 6, 10)

verifying a probability distribution with variable arguments sums to 1

I was wondering how you would write a method in Scala that takes a function f and a list of arguments args where each arg is a range. Suppose I have three arguments (Range(0,2), Range(0,10), and Range(1, 5)). Then I want to iterate over f with all the possibilities of those three arguments.
var sum = 0.0
for (a <- arg(0)) {
for (b <- arg(1)) {
for (c <- arg(2)) {
sum += f(a, b, c)
}
}
}
However, I want this method to work for functions with a variable number of arguments. Is this possible?
Edit: is there any way to do this when the function does not take a list, but rather takes a standard parameter list or is curried?
That's a really good question!
You want to run flatMap in sequence over a list of elements of arbitrary size. When you don't know how long your list is, you can process it with recursion, or equivalently, with a fold.
scala> def sequence[A](lss: List[List[A]]) = lss.foldRight(List(List[A]())) {
| (m, n) => for (x <- m; xs <- n) yield x :: xs
| }
scala> sequence(List(List(1, 2), List(4, 5), List(7)))
res2: List[List[Int]] = List(List(1, 4, 7), List(1, 5, 7), List(2, 4, 7), List(2
, 5, 7))
(If you can't figure out the code, don't worry, learn how to use Hoogle and steal it from Haskell)
You can do this with Scalaz (in general it starts with a F[G[X]] and returns a G[F[X]], given that the type constructors G and F have the Traverse and Applicative capabilities respectively.
scala> import scalaz._
import scalaz._
scala> import Scalaz._
import Scalaz._
scala> List(List(1, 2), List(4, 5), List(7)).sequence
res3: List[List[Int]] = List(List(1, 4, 7), List(1, 5, 7), List(2, 4, 7), List(2
, 5, 7))
scala> Seq(some(1), some(2)).sequence
res4: Option[Seq[Int]] = Some(List(1, 2))
scala> Seq(some(1), none[Int]).sequence
res5: Option[Seq[Int]] = None
That would more or less do the job (without applying f, which you can do separately)
def crossProduct[A](xxs: Seq[A]*) : Seq[Seq[A]]
= xxs.foldLeft(Vector(Vector[A]())){(res, xs) =>
for(r <- res; x <- xs) yield r :+ x
}
You can then just map your function on that. I'm not sure it's a very efficient implementation though.
That's the answer from recursive perspective. Unfortunately, not so short as others.
def foo(f: List[Int] => Int, args: Range*) = {
var sum = 0.0
def rec(ranges: List[Range], ints: List[Int]): Unit = {
if (ranges.length > 0)
for (i <- ranges.head)
rec(ranges.tail, i :: ints)
else
sum += f(ints)
}
rec(args.toList, List[Int]())
sum
}
Have a look at this answer. I use this code for exactly this purpose. It's slightly optimized. I think I could produce a faster version if you need one.