Related
So I'm following http://jelv.is/blog/Lazy-Dynamic-Programming/ and implementing the Fibonacci example in Scala. Here is my implementation:
class Lazy[T] (expr : => T) {
lazy val value = expr
def apply(): T = value
}
object Lazy{ def apply[T](expr : => T) = new Lazy({expr}) }
def fib(n: Int): Int = {
def doFib(i: Int): Lazy[Int] = Lazy {
if (i <= 2) 1
else fibs(i - 1)() + fibs(i - 2)()
}
lazy val fibs = Array.tabulate[Lazy[Int]](n)(doFib)
doFib(n).value
}
fib(5)
In this case, fib(5) correctly returns result 5.
Then I want to see if Lazy[T] can be made into a monad by trying the following code, which results in StackOverflow runtime error:
class Lazy[T] (expr : => T) {
lazy val value = expr
def apply(): T = value
def flatMap[A](f: T => Lazy[A]): Lazy[A] = Lazy { f(value).value }
def map[A](f: T => A): Lazy[A] = Lazy { f(value) }
}
object Lazy{ def apply[T](expr : => T) = new Lazy({expr}) }
def fib(n: Int): Int = {
def doFib(i: Int): Lazy[Int] =
if (i <= 2) Lazy(1)
else for {
a <- fibs(i - 1)
b <- fibs(i - 2)
} yield a + b
lazy val fibs = Array.tabulate[Lazy[Int]](n)(doFib)
doFib(n).value
}
fib(5)
It appears that fibs(i - 1) is calculated too early, which results in infinite recursion. I wonder if there is a for comprehension syntax that's equivalent to the first code snippet?
You are right, "fibs(i - 1) is calculated too early". It is evaluated immediately when you call doFib, because the doFib(i) needs fibs(i - 1) in order to be able to return anything, which in turn needs the return value of doFib(i - 1) and so on, so that the recursion unfolds completely while you are constructing the array of lazy ints (before you invoke doFib(n).value).
If you want it lazy, then return a Lazy that does not require immediate evaluation of fibs(i - 1):
class Lazy[T] (expr : => T) {
lazy val value = expr
def apply(): T = value
def flatMap[A](f: T => Lazy[A]): Lazy[A] = Lazy { f(value).value }
def map[A](f: T => A): Lazy[A] = Lazy { f(value) }
}
object Lazy{ def apply[T](expr : => T) = new Lazy({expr}) }
def fib(n: Int): Int = {
def doFib(i: Int): Lazy[Int] =
if (i <= 2) Lazy(1)
else Lazy{ (for {
a <- fibs(i - 1)
b <- fibs(i - 2)
} yield a + b).value
}
lazy val fibs = Array.tabulate[Lazy[Int]](n)(doFib)
doFib(n).value
}
println(fib(40)) // 102334155
Alternatively, you can wrap the whole if-else in a Lazy:
def doFib(i: Int): Lazy[Int] = Lazy {
if (i <= 2) 1
else (for {
a <- fibs(i - 1)
b <- fibs(i - 2)
} yield a + b).value
}
This produces the same expected result.
I want to write some mergesort function.
How to supply Ordering[T] to merge subfunction?
The overall structure of application is the following:
object Main extends App {
...
val array: Array[Int] = string.split(' ').map(_.toInt)
def mergesort[T](seq: IndexedSeq[T]): IndexedSeq[T] = {
def mergesortWithIndexes(seq: IndexedSeq[T],
startIdx: Int, endIdx: Int): IndexedSeq[T] = {
import Helpers.append
val seqLength = endIdx - startIdx
val splitLength = seq.length / 2
val (xs, ys) = seq.splitAt(splitLength)
val sortXs = mergesortWithIndexes(xs, startIdx, startIdx + seqLength)
val sortYs = mergesortWithIndexes(ys, startIdx + seqLength, endIdx)
def merge(sortXs: IndexedSeq[T], sortYs: IndexedSeq[T],
writeFun: Iterable[CharSequence] => Path)(ord: math.Ordering[T]): IndexedSeq[T] = {
...
while (firstIndex < firstLength || secondIndex < secondLength) {
if (firstIndex == firstLength)
buffer ++ sortYs
else if (secondIndex == secondLength)
buffer ++ sortXs
else {
if (ord.lteq(minFirst, minSecond)) {
...
} else {
...
}
}
}
buffer.toIndexedSeq
}
merge(sortXs, sortYs, append(output))
}
mergesortWithIndexes(seq, 0, seq.length)
}
val outSeq = mergesort(array)
Helpers.write(output)(Vector(outSeq.mkString(" ")))
}
I want to have general merge() function definition, but in application I use IndexedSeq[Int] and thus expecting pass predefined Ordering[Int].
Adding implicit Ordering[T] parameter to the outermost function should fix the problem, and passing non Ordering[T] arguments will result in compile error.
Scala's sort functions do the same thing: https://github.com/scala/scala/blob/2.12.x/src/library/scala/collection/SeqLike.scala#L635
def mergesort[T](seq: IndexedSeq[T])(implicit ord: math.Ordering[T]): IndexedSeq[T] = {
I was wondering how to go about adding a 'partitionCount' method to Lists, e.g.:
(not tested, shamelessly based on List.scala):
Do I have to create my own sub-class and an implicit type converter?
(My original attempt had a lot of problems, so here is one based on #Easy's answer):
class MyRichList[A](targetList: List[A]) {
def partitionCount(p: A => Boolean): (Int, Int) = {
var btrue = 0
var bfalse = 0
var these = targetList
while (!these.isEmpty) {
if (p(these.head)) { btrue += 1 } else { bfalse += 1 }
these = these.tail
}
(btrue, bfalse)
}
}
and here is a little more general version that's good for Seq[...]:
implicit def seqToRichSeq[T](s: Seq[T]) = new MyRichSeq(s)
class MyRichList[A](targetList: List[A]) {
def partitionCount(p: A => Boolean): (Int, Int) = {
var btrue = 0
var bfalse = 0
var these = targetList
while (!these.isEmpty) {
if (p(these.head)) { btrue += 1 } else { bfalse += 1 }
these = these.tail
}
(btrue, bfalse)
}
}
You can use implicit conversion like this:
implicit def listToMyRichList[T](l: List[T]) = new MyRichList(l)
class MyRichList[T](targetList: List[T]) {
def partitionCount(p: T => Boolean): (Int, Int) = ...
}
and instead of this you need to use targetList. You don't need to extend List. In this example I create simple wrapper MyRichList that would be used implicitly.
You can generalize wrapper further, by defining it for Traversable, so that it will work for may other collection types and not only for Lists:
implicit def listToMyRichTraversable[T](l: Traversable[T]) = new MyRichTraversable(l)
class MyRichTraversable[T](target: Traversable[T]) {
def partitionCount(p: T => Boolean): (Int, Int) = ...
}
Also note, that implicit conversion would be used only if it's in scope. This means, that you need to import it (unless you are using it in the same scope where you have defined it).
As already pointed out by Easy Angel, use implicit conversion:
implicit def listTorichList[A](input: List[A]) = new RichList(input)
class RichList[A](val source: List[A]) {
def partitionCount(p: A => Boolean): (Int, Int) = {
val partitions = source partition(p)
(partitions._1.size, partitions._2.size)
}
}
Also note that you can easily define partitionCount in terms of partinion. Then you can simply use:
val list = List(1, 2, 3, 5, 7, 11)
val (odd, even) = list partitionCount {_ % 2 != 0}
If you are curious how it works, just remove implicit keyword and call the list2richList conversion explicitly (this is what the compiler does transparently for you when implicit is used).
val (odd, even) = list2richList(list) partitionCount {_ % 2 != 0}
Easy Angel is right, but the method seems pretty useless. You have already count in order to get the number of "positives", and of course the number of "negatives" is size minus count.
However, to contribute something positive, here a more functional version of your original method:
def partitionCount[A](iter: Traversable[A], p: A => Boolean): (Int, Int) =
iter.foldLeft ((0,0)) { ((x,y), a) => if (p(a)) (x + 1,y) else (x, y + 1)}
Recently, I wrote an iterator for a cartesian product of Anys, and started with a List of List, but recognized, that I can easily switch to the more abstract trait Seq.
I know, you like to see the code. :)
class Cartesian (val ll: Seq[Seq[_]]) extends Iterator [Seq[_]] {
def combicount: Int = (1 /: ll) (_ * _.length)
val last = combicount
var iter = 0
override def hasNext (): Boolean = iter < last
override def next (): Seq[_] = {
val res = combination (ll, iter)
iter += 1
res
}
def combination (xx: Seq [Seq[_]], i: Int): List[_] = xx match {
case Nil => Nil
case x :: xs => x (i % x.length) :: combination (xs, i / x.length)
}
}
And a client of that class:
object Main extends Application {
val illi = new Cartesian (List ("abc".toList, "xy".toList, "AB".toList))
// val ivvi = new Cartesian (Vector (Vector (1, 2, 3), Vector (10, 20)))
val issi = new Cartesian (Seq (Seq (1, 2, 3), Seq (10, 20)))
// val iaai = new Cartesian (Array (Array (1, 2, 3), Array (10, 20)))
(0 to 5).foreach (dummy => println (illi.next ()))
// (0 to 5).foreach (dummy => println (issi.next ()))
}
/*
List(a, x, A)
List(b, x, A)
List(c, x, A)
List(a, y, A)
List(b, y, A)
List(c, y, A)
*/
The code works well for Seq and Lists (which are Seqs), but of course not for Arrays or Vector, which aren't of type Seq, and don't have a cons-method '::'.
But the logic could be used for such collections too.
I could try to write an implicit conversion to and from Seq for Vector, Array, and such, or try to write an own, similar implementation, or write an Wrapper, which transforms the collection to a Seq of Seq, and calls 'hasNext' and 'next' for the inner collection, and converts the result to an Array, Vector or whatever. (I tried to implement such workarounds, but I have to recognize: it's not that easy. For a real world problem I would probably rewrite the Iterator independently.)
However, the whole thing get's a bit out of control if I have to deal with Arrays of Lists or Lists of Arrays and other mixed cases.
What would be the most elegant way to write the algorithm in the broadest, possible way?
There are two solutions. The first is to not require the containers to be a subclass of some generic super class, but to be convertible to one (by using implicit function arguments). If the container is already a subclass of the required type, there's a predefined identity conversion which only returns it.
import collection.mutable.Builder
import collection.TraversableLike
import collection.generic.CanBuildFrom
import collection.mutable.SeqLike
class Cartesian[T, ST[T], TT[S]](val ll: TT[ST[T]])(implicit cbf: CanBuildFrom[Nothing, T, ST[T]], seqLike: ST[T] => SeqLike[T, ST[T]], traversableLike: TT[ST[T]] => TraversableLike[ST[T], TT[ST[T]]] ) extends Iterator[ST[T]] {
def combicount (): Int = (1 /: ll) (_ * _.length)
val last = combicount - 1
var iter = 0
override def hasNext (): Boolean = iter < last
override def next (): ST[T] = {
val res = combination (ll, iter, cbf())
iter += 1
res
}
def combination (xx: TT[ST[T]], i: Int, builder: Builder[T, ST[T]]): ST[T] =
if (xx.isEmpty) builder.result
else combination (xx.tail, i / xx.head.length, builder += xx.head (i % xx.head.length) )
}
This sort of works:
scala> new Cartesian[String, Vector, Vector](Vector(Vector("a"), Vector("xy"), Vector("AB")))
res0: Cartesian[String,Vector,Vector] = empty iterator
scala> new Cartesian[String, Array, Array](Array(Array("a"), Array("xy"), Array("AB")))
res1: Cartesian[String,Array,Array] = empty iterator
I needed to explicitly pass the types because of bug https://issues.scala-lang.org/browse/SI-3343
One thing to note is that this is better than using existential types, because calling next on the iterator returns the right type, and not Seq[Any].
There are several drawbacks here:
If the container is not a subclass of the required type, it is converted to one, which costs in performance
The algorithm is not completely generic. We need types to be converted to SeqLike or TraversableLike only to use a subset of functionality these types offer. So making a conversion function can be tricky.
What if some capabilities can be interpreted differently in different contexts? For example, a rectangle has two 'length' properties (width and height)
Now for the alternative solution. We note that we don't actually care about the types of collections, just their capabilities:
TT should have foldLeft, get(i: Int) (to get head/tail)
ST should have length, get(i: Int) and a Builder
So we can encode these:
trait HasGet[T, CC[_]] {
def get(cc: CC[T], i: Int): T
}
object HasGet {
implicit def seqLikeHasGet[T, CC[X] <: SeqLike[X, _]] = new HasGet[T, CC] {
def get(cc: CC[T], i: Int): T = cc(i)
}
implicit def arrayHasGet[T] = new HasGet[T, Array] {
def get(cc: Array[T], i: Int): T = cc(i)
}
}
trait HasLength[CC] {
def length(cc: CC): Int
}
object HasLength {
implicit def seqLikeHasLength[CC <: SeqLike[_, _]] = new HasLength[CC] {
def length(cc: CC) = cc.length
}
implicit def arrayHasLength[T] = new HasLength[Array[T]] {
def length(cc: Array[T]) = cc.length
}
}
trait HasFold[T, CC[_]] {
def foldLeft[A](cc: CC[T], zero: A)(op: (A, T) => A): A
}
object HasFold {
implicit def seqLikeHasFold[T, CC[X] <: SeqLike[X, _]] = new HasFold[T, CC] {
def foldLeft[A](cc: CC[T], zero: A)(op: (A, T) => A): A = cc.foldLeft(zero)(op)
}
implicit def arrayHasFold[T] = new HasFold[T, Array] {
def foldLeft[A](cc: Array[T], zero: A)(op: (A, T) => A): A = {
var i = 0
var result = zero
while (i < cc.length) {
result = op(result, cc(i))
i += 1
}
result
}
}
}
(strictly speaking, HasFold is not required since its implementation is in terms of length and get, but i added it here so the algorithm will translate more cleanly)
now the algorithm is:
class Cartesian[T, ST[_], TT[Y]](val ll: TT[ST[T]])(implicit cbf: CanBuildFrom[Nothing, T, ST[T]], stHasLength: HasLength[ST[T]], stHasGet: HasGet[T, ST], ttHasFold: HasFold[ST[T], TT], ttHasGet: HasGet[ST[T], TT], ttHasLength: HasLength[TT[ST[T]]]) extends Iterator[ST[T]] {
def combicount (): Int = ttHasFold.foldLeft(ll, 1)((a,l) => a * stHasLength.length(l))
val last = combicount - 1
var iter = 0
override def hasNext (): Boolean = iter < last
override def next (): ST[T] = {
val res = combination (ll, 0, iter, cbf())
iter += 1
res
}
def combination (xx: TT[ST[T]], j: Int, i: Int, builder: Builder[T, ST[T]]): ST[T] =
if (ttHasLength.length(xx) == j) builder.result
else {
val head = ttHasGet.get(xx, j)
val headLength = stHasLength.length(head)
combination (xx, j + 1, i / headLength, builder += stHasGet.get(head, (i % headLength) ))
}
}
And use:
scala> new Cartesian[String, Vector, List](List(Vector("a"), Vector("xy"), Vector("AB")))
res6: Cartesian[String,Vector,List] = empty iterator
scala> new Cartesian[String, Array, Array](Array(Array("a"), Array("xy"), Array("AB")))
res7: Cartesian[String,Array,Array] = empty iterator
Scalaz probably has all of this predefined for you, unfortunately, I don't know it well.
(again I need to pass the types because inference doesn't infer the right kind)
The benefit is that the algorithm is now completely generic and that there is no need for implicit conversions from Array to WrappedArray in order for it to work
Excercise: define for tuples ;-)
I would like to add to all collections where it makes sense, an argMax method.
How to do it? Use implicits?
On Scala 2.8, this works:
val list = List(1, 2, 3)
def f(x: Int) = -x
val argMax = list max (Ordering by f)
As pointed by mkneissl, this does not return the set of maximum points. Here's an alternate implementation that does, and tries to reduce the number of calls to f. If calls to f don't matter that much, see mkneissl's answer. Also, note that his answer is curried, which provides superior type inference.
def argMax[A, B: Ordering](input: Iterable[A], f: A => B) = {
val fList = input map f
val maxFList = fList.max
input.view zip fList filter (_._2 == maxFList) map (_._1) toSet
}
scala> argMax(-2 to 2, (x: Int) => x * x)
res15: scala.collection.immutable.Set[Int] = Set(-2, 2)
The argmax function (as I understand it from Wikipedia)
def argMax[A,B](c: Traversable[A])(f: A=>B)(implicit o: Ordering[B]): Traversable[A] = {
val max = (c map f).max(o)
c filter { f(_) == max }
}
If you really want, you can pimp it onto the collections
implicit def enhanceWithArgMax[A](c: Traversable[A]) = new {
def argMax[B](f: A=>B)(implicit o: Ordering[B]): Traversable[A] = ArgMax.argMax(c)(f)(o)
}
and use it like this
val l = -2 to 2
assert (argMax(l)(x => x*x) == List(-2,2))
assert (l.argMax(x => x*x) == List(-2,2))
(Scala 2.8)
Yes, the usual way would be to use the 'pimp my library' pattern to decorate your collection. For example (N.B. just as illustration, not meant to be a correct or working example):
trait PimpedList[A] {
val l: List[A]
//example argMax, not meant to be correct
def argMax[T <% Ordered[T]](f:T => T) = {error("your definition here")}
}
implicit def toPimpedList[A](xs: List[A]) = new PimpedList[A] {
val l = xs
}
scala> def f(i:Int):Int = 10
f: (i: Int) Int
scala> val l = List(1,2,3)
l: List[Int] = List(1, 2, 3)
scala> l.argMax(f)
java.lang.RuntimeException: your definition here
at scala.Predef$.error(Predef.scala:60)
at PimpedList$class.argMax(:12)
//etc etc...
Nice and easy ? :
val l = List(1,0,10,2)
l.zipWithIndex.maxBy(x => x._1)._2
You can add functions to an existing API in Scala by using the Pimp my Library pattern. You do this by defining an implicit conversion function. For example, I have a class Vector3 to represent 3D vectors:
class Vector3 (val x: Float, val y: Float, val z: Float)
Suppose I want to be able to scale a vector by writing something like: 2.5f * v. I can't directly add a * method to class Float ofcourse, but I can supply an implicit conversion function like this:
implicit def scaleVector3WithFloat(f: Float) = new {
def *(v: Vector3) = new Vector3(f * v.x, f * v.y, f * v.z)
}
Note that this returns an object of a structural type (the new { ... } construct) that contains the * method.
I haven't tested it, but I guess you could do something like this:
implicit def argMaxImplicit[A](t: Traversable[A]) = new {
def argMax() = ...
}
Here's a way of doing so with the implicit builder pattern. It has the advantage over the previous solutions that it works with any Traversable, and returns a similar Traversable. Sadly, it's pretty imperative. If anyone wants to, it could probably be turned into a fairly ugly fold instead.
object RichTraversable {
implicit def traversable2RichTraversable[A](t: Traversable[A]) = new RichTraversable[A](t)
}
class RichTraversable[A](t: Traversable[A]) {
def argMax[That, C](g: A => C)(implicit bf : scala.collection.generic.CanBuildFrom[Traversable[A], A, That], ord:Ordering[C]): That = {
var minimum:C = null.asInstanceOf[C]
val repr = t.repr
val builder = bf(repr)
for(a<-t){
val test: C = g(a)
if(test == minimum || minimum == null){
builder += a
minimum = test
}else if (ord.gt(test, minimum)){
builder.clear
builder += a
minimum = test
}
}
builder.result
}
}
Set(-2, -1, 0, 1, 2).argmax(x=>x*x) == Set(-2, 2)
List(-2, -1, 0, 1, 2).argmax(x=>x*x) == List(-2, 2)
Here's a variant loosely based on #Daniel's accepted answer that also works for Sets.
def argMax[A, B: Ordering](input: GenIterable[A], f: A => B) : GenSet[A] = argMaxZip(input, f) map (_._1) toSet
def argMaxZip[A, B: Ordering](input: GenIterable[A], f: A => B): GenIterable[(A, B)] = {
if (input.isEmpty) Nil
else {
val fPairs = input map (x => (x, f(x)))
val maxF = fPairs.map(_._2).max
fPairs filter (_._2 == maxF)
}
}
One could also do a variant that produces (B, Iterable[A]), of course.
Based on other answers, you can pretty easily combine the strengths of each (minimal calls to f(), etc.). Here we have an implicit conversion for all Iterables (so they can just call .argmax() transparently), and a stand-alone method if for some reason that is preferred. ScalaTest tests to boot.
class Argmax[A](col: Iterable[A]) {
def argmax[B](f: A => B)(implicit ord: Ordering[B]): Iterable[A] = {
val mapped = col map f
val max = mapped max ord
(mapped zip col) filter (_._1 == max) map (_._2)
}
}
object MathOps {
implicit def addArgmax[A](col: Iterable[A]) = new Argmax(col)
def argmax[A, B](col: Iterable[A])(f: A => B)(implicit ord: Ordering[B]) = {
new Argmax(col) argmax f
}
}
class MathUtilsTests extends FunSuite {
import MathOps._
test("Can argmax with unique") {
assert((-10 to 0).argmax(_ * -1).toSet === Set(-10))
// or alternate calling syntax
assert(argmax(-10 to 0)(_ * -1).toSet === Set(-10))
}
test("Can argmax with multiple") {
assert((-10 to 10).argmax(math.pow(_, 2)).toSet === Set(-10, 10))
}
}