Related
After you get a basic idea, coding dynamic programming (DP) problems in imperative style is pretty straightforward, at least for simpler DP problems. It usually involves some form of table, that we iteratively fill based on some formula. This is pretty much all there is for implementing bottom-up DP.
Let's take Longest Increasing Subsequence (LIS) as a simple and common example of a problem that can be solved with bottom-up DP algorithm.
C++ implementation is straightforward (not tested):
// Where A is input (std::vector<int> of size n)
std::vector<int> DP(n, 0);
for(int i = 0; i < n; i++) {
DP[i] = 1;
for(int j = 0; j < i; j++) {
if (A[i] > A[j]) {
DP[i] = std::max(DP[i], DP[j] + 1);
}
}
}
We've just described "how to fill DP" which is exactly what imperative programming is.
If we decide to describe "what DP is", which is kind of a more FP way to think about it, it gets a bit more complicated.
Here's an example Scala code for this problem:
// Where A is input
val DP = A.zipWithIndex.foldLeft(Seq[Int]()) {
case (_, (_, 0)) => Seq(1)
case (d, (a, _)) =>
d :+ d.zipWithIndex.map { case (dj, j) =>
dj + (if (a > A(j)) 1 else 0)
}.max
}
To be quite honest, this Scala implementation doesn't seem that much idiomatic. It's just a translation of imperative solution, with immutability added to it.
I'm curious, what is a general FP way to deal with things like this?
I don't know much about DP but I have a few observations that I hope will contribute to the conversation.
First, your example Scala code doesn't appear to solve the LIS problem. I plugged in the Van der Corput sequence as found on the Wikipedia page and did not get the designated result.
Working through the problem this is the solution I came up with.
def lis(a: Seq[Int], acc: Seq[Int] = Seq.empty[Int]): Int =
if (a.isEmpty) acc.length
else if (acc.isEmpty) lis(a.tail, Seq(a.head)) max lis(a.tail)
else if (a.head > acc.last) lis(a.tail, acc :+ a.head) max lis(a.tail, acc)
else lis(a.tail, acc)
lis(Seq(0, 8, 4, 12, 2, 10, 6, 14, 1, 9, 5, 13, 3, 11, 7, 15)) // res0: Int = 6
This can be adjusted to return the subsequence itself, and I'm sure it can be tweaked for better performance.
As for memoization, it's not hard to roll-your-own on an as-needed basis. Here's the basic outline to memoize any arity-2 function.
def memo[A,B,R](f: (A,B)=>R): ((A,B)=>R) = {
val cache = new collection.mutable.WeakHashMap[(A,B),R]
(a:A,b:B) => cache.getOrElseUpdate((a,b),f(a,b))
}
With this I can create a memoized version of some often called method/function.
val myfunc = memo{(s:String, i:Int) => s.length > i}
myfunc("bsxxlakjs",7) // res0: Boolean = true
Note: It used to be that WeakHashMap was recommended so that the cache could drop lesser used elements in memory-challenged environments. I don't know if that's still the case.
Have only recently started learning Scala and am trying to delve into Functional Programming. I have seen many of the posts on Selection Sort Functional style; but am not totally been able to understand all the solutions that have been given. My Scala skills are still Nascent.
I have written a piece of Scala code using tail recursion and would appreciate any feedback on the style. Does it look like Functional Programming? Is there a way to make this better or make it more functional?
import scala.annotation.tailrec
object FuncSelectionSort {
/**
* Selection Sort - Trying Functional Style
*/
def sort(a: Array[Int]) = {
val b: Array[Int] = new Array[Int](a.size)
Array.copy(a, 0, b, 0, a.size)
// Function to swap elements
def exchange(i: Int, j: Int): Unit = {
val k = b(i);
b(i) = b(j);
b(j) = k;
}
#tailrec
def helper(b: Array[Int], n: Int): Array[Int] = {
if (n == b.length-1) return b
else {
val head = b(n);
val minimumInTail = b.slice(n, b.length).min;
if (head > minimumInTail) {
val minimumInTailIndex = b.slice(n, b.length).indexOf(minimumInTail);
exchange(n, minimumInTailIndex + n);
}
helper(b, n + 1)
}
}
helper(b, 0)
}
}
The logic that I have tried to adopt is fairly simple. I start with the first index of the Array and find the minimum from the rest. But instead of passing the Array.tail for the next recursion; I pass in the full array and check a slice, where each slice is one smaller than the previous recursion slice.
For example,
If Array(10, 4, 6, 9, 3, 5)
First pass -> head = 10, slice = 4,6,9,3,5
First pass -> head = 4, slice = 6,9,3,5
I feel it looks the same as passing the tail, but I wanted to try and slice and see if it works the same way.
Appreciate your help.
For detailed feedback on working code, you should better go to codereview; however, I can say one thing: namely, in-place sorting arrays is per se not a good example of functional programming. This is because we purists don't like mutability, as it doesn't fit together well with recursion over data -- especially your mixing of recursion and mutation is not really good style, I'd say (and hard to read).
One clean variant would be to copy the full original array, and use in-place selection sort implemented as normal imperative code (with loops and in-place swap). Encapsulated in a function, this is pure to the outside. This pattern is commonly used in the standard library; cf. List.scala.
The other variant, and probably more instructive for learning immutable programming, is to use an immutable recursive algorithm over linked lists:
def sorted(a: List[Int]): List[Int] = a match {
case Nil => Nil
case xs => xs.min :: sorted(xs.diff(List(xs.min)))
}
From that style of programming, you'll learn much more about functional thinking (leaving aside efficiency though). Exercise: transform that code into tail-recursion.
(And actually, insertion sort works nicer with this pattern, since you don't have to "remove" at every step, but can build up a sorted linked list; you might try to implement that, too).
I'm new to Scala and was just reading Scala By Example. In chapter 2, the author has 2 different versions of Quicksort.
One is imperative style:
def sort(xs: Array[Int]) {
def swap(i: Int, j: Int) {
val t = xs(i); xs(i) = xs(j); xs(j) = t
}
def sort1(l: Int, r: Int) {
val pivot = xs((l + r) / 2)
var i = l; var j = r
while (i <= j) {
while (xs(i) < pivot) i += 1
while (xs(j) > pivot) j -= 1
if (i <= j) {
swap(i, j)
i += 1
j -= 1
}
}
if (l < j) sort1(l, j)
if (j < r) sort1(i, r)
}
sort1(0, xs.length - 1)
}
One is functional style:
def sort(xs: Array[Int]): Array[Int] = {
if (xs.length <= 1) xs
else {
val pivot = xs(xs.length / 2)
Array.concat(
sort(xs filter (pivot >)),
xs filter (pivot ==),
sort(xs filter (pivot <)))
}
}
The obvious advantage the functional style has over imperative style is conciseness. But what about performance? Since it uses recursion, do we pay for the performance penalty just like we do in other imperative languages like C? Or, Scala being a hybrid language, the "Scala way" (functional) is preferred, thus more efficient.
Note: The author did mention the functional style does use more memory.
It depends. If you look in the Scala sources, there is often an imperative style used "under the hood" in order to be performant - but in many cases exactly these tweaks allow you write performant functional code. So usually you can come up with a functional solution that is fast enough, but you must be careful and know what you do (especially concerning your data structures). E.g. the array concat in the second example is not nice, but probably not too bad - but using Lists here and concat them with ::: would be overkill.
But that is not more than educated guessing if you don't actually measure the performance. In complex projects it's really hard to predict the performance, especially as things like object creation and method calls get more and more optimized by the compiler and the JVM.
I'd suggest to start out with the functional style. If it is too slow, profile it. Usually there is a better functional solution. If not, you can use the imperative style (or a mix of both) as a last resort.
I want to generate a list of integers corresponding to a list of generators in ScalaCheck.
import org.scalacheck._
import Arbitrary.arbitrary
val smallInt = Gen.choose(0,10)
val bigInt = Gen.choose(1000, 1000000)
val zeroOrOneInt = Gen.choose(0, 1)
val smallEvenInt = smallInt suchThat (_ % 2 == 0)
val gens = List(smallInt, bigInt, zeroOrOneInt, smallEvenInt)
//val listGen: Gen[Int] = ??
//println(listGen.sample) //should print something like List(2, 2000, 0, 6)
For the given gens, I would like to create a generator listGen whose valid sample can be List(2, 2000, 0, 6).
Here is my first attempt using tuples.
val gensTuple = (smallInt, bigInt, zeroOrOneInt, smallEvenInt)
val tupleGen = for {
a <- gensTuple._1
b <- gensTuple._2
c <- gensTuple._3
d <- gensTuple._4
} yield (a, b, c, d)
println(tupleGen.sample) // prints Some((1,318091,0,6))
This works, but I don't want to use tuples since the list of generators(gens) is created dynamically
and the size of the list is not fixed. Is there a way to do it with Lists?
I want the use the generator of the list(listGen) in scalacheck forAll property checking.
This looks like a toy problem but this is
the best I could do to create a standalone snippet reproducing the actual issue I am
facing.
How about using the Gen.sequence method? It transforms an Iterable[Gen[T]] into a Gen[C[T]], where C can be List:
def sequence[C[_],T](gs: Iterable[Gen[T]])(implicit b: Buildable[T,C]): Gen[C[T]] =
...
Just use Gen.sequence, but be careful as it will try to return a java.util.ArrayList[T] if you don't fully parameterize it (bug).
Full working example:
def genIntList(): Gen[List[Int]] = {
val gens = List(Gen.chooseNum(1, 2), Gen.chooseNum(3, 4))
Gen.sequence[List[Int], Int](gens)
}
println(genIntList.sample.get) // prints: List(1,4)
EDIT: Please disregard, this doesn't answer the asker's question
I can't comment on posts yet, so I'll have to venture a guess here. I presume the function 'sample' applies to the generators
Any reason why you can't do:
gens map (t=>t.sample)
For a more theoretical answer: the method you want is traverse, which is equivalent to sequence compose map although it might be more efficient. It is of the general form:
def traverse[C[_]: Traverse, F[_]: Applicative, A, B](f: A => F[B], t: C[A]): F[C[B]]
It behaves like map but allows you to carry around some extra Applicative structure during the traversal, sequencing it along the way.
I understand Ruby and Python's yield. What does Scala's yield do?
I think the accepted answer is great, but it seems many people have failed to grasp some fundamental points.
First, Scala's for comprehensions are equivalent to Haskell's do notation, and it is nothing more than a syntactic sugar for composition of multiple monadic operations. As this statement will most likely not help anyone who needs help, let's try again… :-)
Scala's for comprehensions is syntactic sugar for composition of multiple operations with map, flatMap and filter. Or foreach. Scala actually translates a for-expression into calls to those methods, so any class providing them, or a subset of them, can be used with for comprehensions.
First, let's talk about the translations. There are very simple rules:
This
for(x <- c1; y <- c2; z <-c3) {...}
is translated into
c1.foreach(x => c2.foreach(y => c3.foreach(z => {...})))
This
for(x <- c1; y <- c2; z <- c3) yield {...}
is translated into
c1.flatMap(x => c2.flatMap(y => c3.map(z => {...})))
This
for(x <- c; if cond) yield {...}
is translated on Scala 2.7 into
c.filter(x => cond).map(x => {...})
or, on Scala 2.8, into
c.withFilter(x => cond).map(x => {...})
with a fallback into the former if method withFilter is not available but filter is. Please see the section below for more information on this.
This
for(x <- c; y = ...) yield {...}
is translated into
c.map(x => (x, ...)).map((x,y) => {...})
When you look at very simple for comprehensions, the map/foreach alternatives look, indeed, better. Once you start composing them, though, you can easily get lost in parenthesis and nesting levels. When that happens, for comprehensions are usually much clearer.
I'll show one simple example, and intentionally omit any explanation. You can decide which syntax was easier to understand.
l.flatMap(sl => sl.filter(el => el > 0).map(el => el.toString.length))
or
for {
sl <- l
el <- sl
if el > 0
} yield el.toString.length
withFilter
Scala 2.8 introduced a method called withFilter, whose main difference is that, instead of returning a new, filtered, collection, it filters on-demand. The filter method has its behavior defined based on the strictness of the collection. To understand this better, let's take a look at some Scala 2.7 with List (strict) and Stream (non-strict):
scala> var found = false
found: Boolean = false
scala> List.range(1,10).filter(_ % 2 == 1 && !found).foreach(x => if (x == 5) found = true else println(x))
1
3
7
9
scala> found = false
found: Boolean = false
scala> Stream.range(1,10).filter(_ % 2 == 1 && !found).foreach(x => if (x == 5) found = true else println(x))
1
3
The difference happens because filter is immediately applied with List, returning a list of odds -- since found is false. Only then foreach is executed, but, by this time, changing found is meaningless, as filter has already executed.
In the case of Stream, the condition is not immediatelly applied. Instead, as each element is requested by foreach, filter tests the condition, which enables foreach to influence it through found. Just to make it clear, here is the equivalent for-comprehension code:
for (x <- List.range(1, 10); if x % 2 == 1 && !found)
if (x == 5) found = true else println(x)
for (x <- Stream.range(1, 10); if x % 2 == 1 && !found)
if (x == 5) found = true else println(x)
This caused many problems, because people expected the if to be considered on-demand, instead of being applied to the whole collection beforehand.
Scala 2.8 introduced withFilter, which is always non-strict, no matter the strictness of the collection. The following example shows List with both methods on Scala 2.8:
scala> var found = false
found: Boolean = false
scala> List.range(1,10).filter(_ % 2 == 1 && !found).foreach(x => if (x == 5) found = true else println(x))
1
3
7
9
scala> found = false
found: Boolean = false
scala> List.range(1,10).withFilter(_ % 2 == 1 && !found).foreach(x => if (x == 5) found = true else println(x))
1
3
This produces the result most people expect, without changing how filter behaves. As a side note, Range was changed from non-strict to strict between Scala 2.7 and Scala 2.8.
It is used in sequence comprehensions (like Python's list-comprehensions and generators, where you may use yield too).
It is applied in combination with for and writes a new element into the resulting sequence.
Simple example (from scala-lang)
/** Turn command line arguments to uppercase */
object Main {
def main(args: Array[String]) {
val res = for (a <- args) yield a.toUpperCase
println("Arguments: " + res.toString)
}
}
The corresponding expression in F# would be
[ for a in args -> a.toUpperCase ]
or
from a in args select a.toUpperCase
in Linq.
Ruby's yield has a different effect.
Yes, as Earwicker said, it's pretty much the equivalent to LINQ's select and has very little to do with Ruby's and Python's yield. Basically, where in C# you would write
from ... select ???
in Scala you have instead
for ... yield ???
It's also important to understand that for-comprehensions don't just work with sequences, but with any type which defines certain methods, just like LINQ:
If your type defines just map, it allows for-expressions consisting of a
single generator.
If it defines flatMap as well as map, it allows for-expressions consisting
of several generators.
If it defines foreach, it allows for-loops without yield (both with single and multiple generators).
If it defines filter, it allows for-filter expressions starting with an if
in the for expression.
Unless you get a better answer from a Scala user (which I'm not), here's my understanding.
It only appears as part of an expression beginning with for, which states how to generate a new list from an existing list.
Something like:
var doubled = for (n <- original) yield n * 2
So there's one output item for each input (although I believe there's a way of dropping duplicates).
This is quite different from the "imperative continuations" enabled by yield in other languages, where it provides a way to generate a list of any length, from some imperative code with almost any structure.
(If you're familiar with C#, it's closer to LINQ's select operator than it is to yield return).
Consider the following for-comprehension
val A = for (i <- Int.MinValue to Int.MaxValue; if i > 3) yield i
It may be helpful to read it out loud as follows
"For each integer i, if it is greater than 3, then yield (produce) i and add it to the list A."
In terms of mathematical set-builder notation, the above for-comprehension is analogous to
which may be read as
"For each integer , if it is greater than , then it is a member of the set ."
or alternatively as
" is the set of all integers , such that each is greater than ."
The keyword yield in Scala is simply syntactic sugar which can be easily replaced by a map, as Daniel Sobral already explained in detail.
On the other hand, yield is absolutely misleading if you are looking for generators (or continuations) similar to those in Python. See this SO thread for more information: What is the preferred way to implement 'yield' in Scala?
Yield is similar to for loop which has a buffer that we cannot see and for each increment, it keeps adding next item to the buffer. When the for loop finishes running, it would return the collection of all the yielded values. Yield can be used as simple arithmetic operators or even in combination with arrays.
Here are two simple examples for your better understanding
scala>for (i <- 1 to 5) yield i * 3
res: scala.collection.immutable.IndexedSeq[Int] = Vector(3, 6, 9, 12, 15)
scala> val nums = Seq(1,2,3)
nums: Seq[Int] = List(1, 2, 3)
scala> val letters = Seq('a', 'b', 'c')
letters: Seq[Char] = List(a, b, c)
scala> val res = for {
| n <- nums
| c <- letters
| } yield (n, c)
res: Seq[(Int, Char)] = List((1,a), (1,b), (1,c), (2,a), (2,b), (2,c), (3,a), (3,b), (3,c))
Hope this helps!!
val aList = List( 1,2,3,4,5 )
val res3 = for ( al <- aList if al > 3 ) yield al + 1
val res4 = aList.filter(_ > 3).map(_ + 1)
println( res3 )
println( res4 )
These two pieces of code are equivalent.
val res3 = for (al <- aList) yield al + 1 > 3
val res4 = aList.map( _+ 1 > 3 )
println( res3 )
println( res4 )
These two pieces of code are also equivalent.
Map is as flexible as yield and vice-versa.
val doubledNums = for (n <- nums) yield n * 2
val ucNames = for (name <- names) yield name.capitalize
Notice that both of those for-expressions use the yield keyword:
Using yield after for is the “secret sauce” that says, “I want to yield a new collection from the existing collection that I’m iterating over in the for-expression, using the algorithm shown.”
taken from here
According to the Scala documentation, it clearly says "yield a new collection from the existing collection".
Another Scala documentation says, "Scala offers a lightweight notation for expressing sequence comprehensions. Comprehensions have the form for (enums) yield e, where enums refers to a semicolon-separated list of enumerators. An enumerator is either a generator which introduces new variables, or it is a filter. "
yield is more flexible than map(), see example below
val aList = List( 1,2,3,4,5 )
val res3 = for ( al <- aList if al > 3 ) yield al + 1
val res4 = aList.map( _+ 1 > 3 )
println( res3 )
println( res4 )
yield will print result like: List(5, 6), which is good
while map() will return result like: List(false, false, true, true, true), which probably is not what you intend.