I am doing the scala coursera course and trying to write an implementation for the 'decode' method of the huffman algorithm. I am new to scala.
Following is my code(so far).
def decode(tree: CodeTree, bits: List[Bit]): List[Char] = {
def innerDecode(subTree: CodeTree, subBits: List[Bit]): List[Char] =
subBits match {
case head :: rest => {
subTree match {
case Leaf(c, w) => c :: innerDecode(tree, rest)
case Fork(left, right, _, _) => {
if ( head == 0) innerDecode(left, rest)
else innerDecode(right, rest)
}
}
}
case Nil => List[Char]()
}
innerDecode(tree, bits)
}
When writing a test e.g. below:
val decoded = decode(t1, List(0, 1))
assert(decoded === List('a','b'))// decoded is returned as List('a') instead of List('a','b')
Gives t1 as: Fork(Leaf('a',2), Leaf('b',3), List('a','b'), 5)
Can someone please advise why the implementation will return just List('a')
The encode for aabbb I assume is:
a -> 0
b -> 1
In Huffman decoding you take one bit of subBits each time you fork, go right or left. In your implementation you do this when you fork and when you reach a Leaf. Each time you reach a letter you throw one additional bit away.
You could see that with few more tests, in your case:
decode(List(0,1)) === List(a)
decode(List(0,0,1)) === List(a,b)
decode(List(0,0,0)) === List(a,a)
decode(List(0,1,0)) === List(a,a)
Or you could use debugger and go trough execution of your code step by step.
I have implemented a recursive method to check if number of parenthesis in a string is valid or not. Here is the code
def balance(chars: List[Char]): Boolean = {
#tailrec
def isValid(newChars: List[Char], difference: Int): Boolean = {
if (newChars.isEmpty) difference == 0
else if (difference < 0) false
else {
var newDifference = difference // Scala IDE gives warning here
if (newChars.head == '(') newDifference = difference + 1
else if (newChars.head == ')') newDifference = difference - 1
isValid(newChars.tail, newDifference)
}
}
isValid(chars, 0)
}
I tested the above code for the following test cases and it works fine So I am only looking for improving the if/else ladder.
println("Testing parenthesis balancing")
assert(balance("(Sachin is learning (scala) and (spark))".toList))
assert(!balance("(Invalid))(expression)".toList))
assert(balance("".toList))
assert(balance("()()".toList))
assert(!balance("{())}{()}".toList))
As mentioned in the code, Scala IDE complains on that line saying
Avoid mutable local variables
I am not really sure how to compute the value of newDifference without using if/else. Other option I could see is directly call isValid method in if/else ladder with computed values of newDifference.
I am still learning Scala So I want to know what could be the best possible way to write this code without mutating the local variable (or any other warning).
People use pattern match for that. That way you can avoid both mutable variables and the "if/else ladder", that makes for horrible "spaghetti code".
def isValid(chars: List[Char], ps: Int = 0) = (chars, ps) match {
case (Nil, _) => ps == 0
case (_, _) if ps < 0 => false
case ('(' :: tail, ps) => isValid(tail, ps + 1)
case (')' :: tail, ps) => isValid(tail, ps - 1)
case (_ :: tail, ps) => isValid(tail, ps)
}
You can write:
val newDifference =
if (newChars.head == '(') difference + 1
else if (newChars.head == ')') difference - 1
else difference;
as if in Scala is an expression, or use match, which would be considered more idiomatic in this case:
val newDifference = newChars.head match {
case '(' => difference + 1
case ')' => difference - 1
case _ => difference
}
The whole function can be converted into a single match on newChars, but I'll leave that to you. See the first example here for some ideas.
I'm a new beginner to Scala, and I'm now learning the for statements. I read this tutorial http://joelabrahamsson.com/learning-scala-part-six-if-statements-and-loops/
And in this tutorial, there is a example,
for (person: Person <- people
if !person.female;
name = person.name;
if name.contains("Ewing"))
println(name)
If compare this for loop to the for loop in Java, is it like
for(person: people) {
if (!person.female) {
String name = person.name;
if (name.contains("Ewing"))
println(name)
}
}
or like this:
for(person: people) {
String name = person.name;
if (!person.female && name.contains("Ewing")) {
println(name)
}
}
Are the operations (in this example, name = person.name;) executed if the first filter condition "if !person.female;" is not satisfied?
Thanks!
To see what the scala compiler generates, compile as scalac -Xprint:typer. It gives:
people.withFilter(((check$ifrefutable$1: typer.Person) => check$ifrefutable$1: #scala.unchecked match {
case (person # (_: Person)) => true
case _ => false
}))
//filter is acting as your if-clause
.withFilter(((person: Person) => person.<female: error>.unary_!)).map(((person: Person) => {
val name = person.name;
scala.Tuple2(person, name)
}))
//Your next if-clause
.withFilter(((x$1) => x$1: #scala.unchecked match {
case scala.Tuple2((person # (_: Person)), (name # _)) => name.contains("Ewing")
}))
//Print each of them
.foreach(((x$2) => x$2: #scala.unchecked match {
case scala.Tuple2((person # (_: Person)), (name # _)) => println(name)
}))
}
}
So in short it as acting as your first mentioned case. But as a concept, it is always recommended to think for-comprehensions as a mapping of map, foreach, flatmap etc.
This is because in many cases while dealing with yield you will need to manage types and thinking in terms of foreach and filter (which in java sense is foreach and if) will not cover all cases. For example, consider below:
scala> for(x <- Option(1);
| u <- scala.util.Left(2)
| ) yield (x,u)
<console>:9: error: value map is not a member of scala.util.Left[Int,Nothing]
u <- scala.util.Left(2)
Above for comprehension uses flatmap and map. Thinking in terms of Java for loops (foreach basically`) will not help in finding the reason.
Scala for comprehension unfolds into combination of map, flatmap and filter. In your case if is actually a filter for all values that appear before it in this "iteration" of the "loop". So if if condition is not satisfied the loop will skip this iteration, so your Java for loops behave the same way as Scala example.
For example try this in REPL:
scala> val l = List(1, 2, 3, 4, 5, 6)
l: List[Int] = List(1, 2, 3, 4, 5, 6)
scala> for (i <- l
| if(i%2 == 0))
| println(i)
2
4
6
scala>
This is equivalent to:
l.filter(_%2 == 0).foreach(println)
Try it out!
for{
x <- 1 to 10
if x % 3 == 0
y = println(f"x=$x")
if x % 2 == 0
} {
println(x)
}
prints:
x=3
x=6
x=9
6
So this means that the y= line is happening before the second if filter.
I think it would not evaluate the follows expressions as it is a common implementation to do so with AND conditions. You can see here
http://www.scala-lang.org/api/current/index.html#scala.Boolean
That scala also has this short circuit implementation when using &&
Scenario:
val col: IndexedSeq[Array[Char]] = for (i <- 1 to n) yield {
val x = for (j <- 1 to m) yield 'x'
x.toArray
}
This is a fairly simple char matrix. toArray used to allow updating.
var west = last.x - 1
while (west >= 0 && arr(last.y)(west) == '.') {
arr(last.y)(west) = ch;
west -= 1;
}
This is updating all . to ch until a non-dot char is found.
Generically, update until stop condition is met, unknown number of steps.
What is the idiomatic equivalent of it?
Conclusion
It's doable, but the trade-off isn't worth it, a lot of performance is lost to expressive syntax when the collection allows updating.
Your wish for a "cleaner, more idiomatic" solution is of course a little fuzzy, because it leaves a lot of room for subjectivity. In general, I'd consider a tail-recursive updating routine more idiomatic, but it might not be "cleaner" if you're more familiar with a non-functional programming style. I came up with this:
#tailrec
def update(arr:List[Char], replace:Char, replacement:Char, result:List[Char] = Nil):List[Char] = arr match {
case `replace` :: tail =>
update(tail, replace, replacement, replacement :: result)
case _ => result.reverse ::: arr
}
This takes one of the inner sequences (assuming a List for easier pattern matching, since Arrays are trivially convertible to lists), and replaces the replace char with the replacement recursively.
You can then use map to update the outer sequence, like so:
col.map { x => update(x, '.', ch) }
Another more reusable alternative is writing your own mapUntil, or using one which is implemented in a supplemental library (Scalaz probably has something like it). The one I came up with looks like this:
def mapUntil[T](input:List[T])(f:(T => Option[T])) = {
#tailrec
def inner(xs:List[T], result:List[T]):List[T] = xs match {
case Nil => Nil
case head :: tail => f(head) match {
case None => (head :: result).reverse ::: tail
case Some(x) => inner(tail, x :: result)
}
}
inner(input, Nil)
}
It does the same as a regular map invocation, except that it stops as soon as the passed function returns None, e.g.
mapUntil(List(1,2,3,4)) {
case x if x >= 3 => None
case x => Some(x-1)
}
Will result in
List[Int] = List(0, 1, 3, 4)
If you want to look at Scalaz, this answer might be a good place to start.
x3ro's answer is the right answer, esp. if you care about performance or are going to be using this operation in multiple places. I would like to add simple solution using only what you find in the collections API:
col.map { a =>
val (l, r) = a.span(_ == '.')
l.map {
case '.' => ch
case x => x
} ++ r
}
What's the best way to terminate a fold early? As a simplified example, imagine I want to sum up the numbers in an Iterable, but if I encounter something I'm not expecting (say an odd number) I might want to terminate. This is a first approximation
def sumEvenNumbers(nums: Iterable[Int]): Option[Int] = {
nums.foldLeft (Some(0): Option[Int]) {
case (Some(s), n) if n % 2 == 0 => Some(s + n)
case _ => None
}
}
However, this solution is pretty ugly (as in, if I did a .foreach and a return -- it'd be much cleaner and clearer) and worst of all, it traverses the entire iterable even if it encounters a non-even number.
So what would be the best way to write a fold like this, that terminates early? Should I just go and write this recursively, or is there a more accepted way?
My first choice would usually be to use recursion. It is only moderately less compact, is potentially faster (certainly no slower), and in early termination can make the logic more clear. In this case you need nested defs which is a little awkward:
def sumEvenNumbers(nums: Iterable[Int]) = {
def sumEven(it: Iterator[Int], n: Int): Option[Int] = {
if (it.hasNext) {
val x = it.next
if ((x % 2) == 0) sumEven(it, n+x) else None
}
else Some(n)
}
sumEven(nums.iterator, 0)
}
My second choice would be to use return, as it keeps everything else intact and you only need to wrap the fold in a def so you have something to return from--in this case, you already have a method, so:
def sumEvenNumbers(nums: Iterable[Int]): Option[Int] = {
Some(nums.foldLeft(0){ (n,x) =>
if ((n % 2) != 0) return None
n+x
})
}
which in this particular case is a lot more compact than recursion (though we got especially unlucky with recursion since we had to do an iterable/iterator transformation). The jumpy control flow is something to avoid when all else is equal, but here it's not. No harm in using it in cases where it's valuable.
If I was doing this often and wanted it within the middle of a method somewhere (so I couldn't just use return), I would probably use exception-handling to generate non-local control flow. That is, after all, what it is good at, and error handling is not the only time it's useful. The only trick is to avoid generating a stack trace (which is really slow), and that's easy because the trait NoStackTrace and its child trait ControlThrowable already do that for you. Scala already uses this internally (in fact, that's how it implements the return from inside the fold!). Let's make our own (can't be nested, though one could fix that):
import scala.util.control.ControlThrowable
case class Returned[A](value: A) extends ControlThrowable {}
def shortcut[A](a: => A) = try { a } catch { case Returned(v) => v }
def sumEvenNumbers(nums: Iterable[Int]) = shortcut{
Option(nums.foldLeft(0){ (n,x) =>
if ((x % 2) != 0) throw Returned(None)
n+x
})
}
Here of course using return is better, but note that you could put shortcut anywhere, not just wrapping an entire method.
Next in line for me would be to re-implement fold (either myself or to find a library that does it) so that it could signal early termination. The two natural ways of doing this are to not propagate the value but an Option containing the value, where None signifies termination; or to use a second indicator function that signals completion. The Scalaz lazy fold shown by Kim Stebel already covers the first case, so I'll show the second (with a mutable implementation):
def foldOrFail[A,B](it: Iterable[A])(zero: B)(fail: A => Boolean)(f: (B,A) => B): Option[B] = {
val ii = it.iterator
var b = zero
while (ii.hasNext) {
val x = ii.next
if (fail(x)) return None
b = f(b,x)
}
Some(b)
}
def sumEvenNumbers(nums: Iterable[Int]) = foldOrFail(nums)(0)(_ % 2 != 0)(_ + _)
(Whether you implement the termination by recursion, return, laziness, etc. is up to you.)
I think that covers the main reasonable variants; there are some other options also, but I'm not sure why one would use them in this case. (Iterator itself would work well if it had a findOrPrevious, but it doesn't, and the extra work it takes to do that by hand makes it a silly option to use here.)
The scenario you describe (exit upon some unwanted condition) seems like a good use case for the takeWhile method. It is essentially filter, but should end upon encountering an element that doesn't meet the condition.
For example:
val list = List(2,4,6,8,6,4,2,5,3,2)
list.takeWhile(_ % 2 == 0) //result is List(2,4,6,8,6,4,2)
This will work just fine for Iterators/Iterables too. The solution I suggest for your "sum of even numbers, but break on odd" is:
list.iterator.takeWhile(_ % 2 == 0).foldLeft(...)
And just to prove that it's not wasting your time once it hits an odd number...
scala> val list = List(2,4,5,6,8)
list: List[Int] = List(2, 4, 5, 6, 8)
scala> def condition(i: Int) = {
| println("processing " + i)
| i % 2 == 0
| }
condition: (i: Int)Boolean
scala> list.iterator.takeWhile(condition _).sum
processing 2
processing 4
processing 5
res4: Int = 6
You can do what you want in a functional style using the lazy version of foldRight in scalaz. For a more in depth explanation, see this blog post. While this solution uses a Stream, you can convert an Iterable into a Stream efficiently with iterable.toStream.
import scalaz._
import Scalaz._
val str = Stream(2,1,2,2,2,2,2,2,2)
var i = 0 //only here for testing
val r = str.foldr(Some(0):Option[Int])((n,s) => {
println(i)
i+=1
if (n % 2 == 0) s.map(n+) else None
})
This only prints
0
1
which clearly shows that the anonymous function is only called twice (i.e. until it encounters the odd number). That is due to the definition of foldr, whose signature (in case of Stream) is def foldr[B](b: B)(f: (Int, => B) => B)(implicit r: scalaz.Foldable[Stream]): B. Note that the anonymous function takes a by name parameter as its second argument, so it need no be evaluated.
Btw, you can still write this with the OP's pattern matching solution, but I find if/else and map more elegant.
Well, Scala does allow non local returns. There are differing opinions on whether or not this is a good style.
scala> def sumEvenNumbers(nums: Iterable[Int]): Option[Int] = {
| nums.foldLeft (Some(0): Option[Int]) {
| case (None, _) => return None
| case (Some(s), n) if n % 2 == 0 => Some(s + n)
| case (Some(_), _) => None
| }
| }
sumEvenNumbers: (nums: Iterable[Int])Option[Int]
scala> sumEvenNumbers(2 to 10)
res8: Option[Int] = None
scala> sumEvenNumbers(2 to 10 by 2)
res9: Option[Int] = Some(30)
EDIT:
In this particular case, as #Arjan suggested, you can also do:
def sumEvenNumbers(nums: Iterable[Int]): Option[Int] = {
nums.foldLeft (Some(0): Option[Int]) {
case (Some(s), n) if n % 2 == 0 => Some(s + n)
case _ => return None
}
}
You can use foldM from cats lib (as suggested by #Didac) but I suggest to use Either instead of Option if you want to get actual sum out.
bifoldMap is used to extract the result from Either.
import cats.implicits._
def sumEven(nums: Stream[Int]): Either[Int, Int] = {
nums.foldM(0) {
case (acc, n) if n % 2 == 0 => Either.right(acc + n)
case (acc, n) => {
println(s"Stopping on number: $n")
Either.left(acc)
}
}
}
examples:
println("Result: " + sumEven(Stream(2, 2, 3, 11)).bifoldMap(identity, identity))
> Stopping on number: 3
> Result: 4
println("Result: " + sumEven(Stream(2, 7, 2, 3)).bifoldMap(identity, identity))
> Stopping on number: 7
> Result: 2
Cats has a method called foldM which does short-circuiting (for Vector, List, Stream, ...).
It works as follows:
def sumEvenNumbers(nums: Stream[Int]): Option[Long] = {
import cats.implicits._
nums.foldM(0L) {
case (acc, c) if c % 2 == 0 => Some(acc + c)
case _ => None
}
}
If it finds a not even element it returns None without computing the rest, otherwise it returns the sum of the even entries.
If you want to keep count until an even entry is found, you should use an Either[Long, Long]
#Rex Kerr your answer helped me, but I needed to tweak it to use Either
def foldOrFail[A,B,C,D](map: B => Either[D, C])(merge: (A, C) => A)(initial: A)(it: Iterable[B]): Either[D, A] = {
val ii= it.iterator
var b= initial
while (ii.hasNext) {
val x= ii.next
map(x) match {
case Left(error) => return Left(error)
case Right(d) => b= merge(b, d)
}
}
Right(b)
}
You could try using a temporary var and using takeWhile. Here is a version.
var continue = true
// sample stream of 2's and then a stream of 3's.
val evenSum = (Stream.fill(10)(2) ++ Stream.fill(10)(3)).takeWhile(_ => continue)
.foldLeft(Option[Int](0)){
case (result,i) if i%2 != 0 =>
continue = false;
// return whatever is appropriate either the accumulated sum or None.
result
case (optionSum,i) => optionSum.map( _ + i)
}
The evenSum should be Some(20) in this case.
You can throw a well-chosen exception upon encountering your termination criterion, handling it in the calling code.
A more beutiful solution would be using span:
val (l, r) = numbers.span(_ % 2 == 0)
if(r.isEmpty) Some(l.sum)
else None
... but it traverses the list two times if all the numbers are even
Just for an "academic" reasons (:
var headers = Source.fromFile(file).getLines().next().split(",")
var closeHeaderIdx = headers.takeWhile { s => !"Close".equals(s) }.foldLeft(0)((i, S) => i+1)
Takes twice then it should but it is a nice one liner.
If "Close" not found it will return
headers.size
Another (better) is this one:
var headers = Source.fromFile(file).getLines().next().split(",").toList
var closeHeaderIdx = headers.indexOf("Close")