I was reading an article at https://medium.com/#aarshkshah1992/scala-functional-memoization-and-lazy-loading-caches-de116f24828
and this part is really interesting
def memoizedIsPrime: Int => Boolean = {
def checkIfPrime(i: Int): Boolean = {
2 to (i - 1) forall (x => i % x != 0)
}
var cache = Map.empty[Int, Boolean]
i => {
if (!cache.contains(i)) {
print(s"Calling isPrime since input ${i} has not been seen before and caching the output")
cache = cache updated(i, checkIfPrime(i))
}
else print(s"Input ${i} has been seen before , returning cached output")
cache(i)
}
}
val isPrime = memoizedIsPrime
Can anyone explain how i is being accessed and the how the code part after => is really working?
Thanks in advance :)
The result of memoizedIsPrime is Int => Boolean, in other words a function that takes an Int and returns a Boolean.
So in the body of the code, after we get some preliminary def and val definitions out of the way, we need a way to reference the incoming Int:
i => ...
OK, so it is decided, we'll refer to the received Int as i. Now we need to produce a Boolean value. And that, of course, is the { } delimited block of code that comes after the => which gets executed every time a new Int arrives.
Related
What is the best way to implement a sorting function that has an internal state received else where?
Something like:
type Sorter[Item] = (Item, Item) => Boolean
type StringSorter = Sorter[String]
def customSorter : StringSorter = (i1,i2) =>
{
val i1Cnt = itemCountMap.get(i1)
val i2Cnt = itemCountMap.get(i2)
if (i1Cnt==None || i2Cnt==None ) {
i1<i2
} else {
i1Cnt.get<i2Cnt.get
}
}
And here is an example:
val l1 = List("a","b","c")
val itemCountMap = Map("a"->2,"b"->3,"c"1)
l1.sortWith(customSorter)
//The returned list will be ["c","a","b"]
I am pretty new to Scala, and in general, lambda functions are not suppose to have states (right?).
Why you ask? 'cause I am using a generic type lists in spark which later, deep in the code of the executors, I want to analyze based on a specific order, and this order may depend on some static list, and I also want to control this order function.
First, this i1Cnt==null will never be true because get on Map never returns null, it returns an Option WHICH IS NOT A NULL, it is very different.
Second, there is no problem with state in a lambda, there is a problem with mutable shared state (and not only in a lambda, but everywhere).
Third, your function would be better if it receives the Map to use instead of relying on a global variable.
Fourth, here is the fixed code.
type Sorter[Item] = (Item, Item) => Boolean
def customSorter(map: Map[String, Int]): Sorter[String] = { (s1, s2) =>
(map.get(s1), map.get(s2)) match {
case (Some(i1), Some(i2)) => i1 < i2
case _ => s1 < s2
}
}
You can see it running here)
I'm trying to write a class where when you call a function defined in the class, it will store it in an array of functions instead of executing it right away, then user calls exec() to execute it:
class TestA(val a: Int, newAction: Option[ArrayBuffer[(Int) => Int]]) {
val action: ArrayBuffer[(Int) => Int] = if (newAction.isEmpty) ArrayBuffer.empty[(Int) => Int] else newAction.get
def add(b: Int): TestA = {action += (a => a + b); new TestA(a, Some(action))}
def exec(): Int = {
var result = 0
action.foreach(r => result += r.apply(a))
result
}
def this(a:Int) = this(a, None)
}
Then this is my test code:
"delayed action" should "delay action till ready" in {
val test = new TestA(3)
val result = test.add(5).add(5)
println(result.exec())
}
This gives me a result of 16 because 3 was passed in twice and got added twice. I guess the easy way for me to solve this problem is to not pass in value for the second round, like change val a: Int to val a: Option[Int]. It helps but it doesn't solve my real problem: letting the second function know the result of the first execution.
Does anyone have a better solution to this?? Or if this is a pattern, can anyone share a tutorial of it?
Just save the result of the action in the 'result' variable (instatiate it with 'a') and use the previous result as input for the current iteration
def exec(): Int = {
var result = a
action.foreach(r => result = r.apply(result))
result
}
or use the more functional oriented solution that does the same
def exec(): Int = {
action.foldLeft(a)((r, f) => f.apply(r))
}
I have the following recursive function in Scala that should return the maximum size integer in the List. Is anyone able to tell me why the largest value is not returned?
def max(xs: List[Int]): Int = {
var largest = xs.head
println("largest: " + largest)
if (!xs.tail.isEmpty) {
var next = xs.tail.head
println("next: " + next)
largest = if (largest > next) largest else next
var remaining = List[Int]()
remaining = largest :: xs.tail.tail
println("remaining: " + remaining)
max(remaining)
}
return largest
}
Print out statements show me that I've successfully managed to bring back the largest value in the List as the head (which was what I wanted) but the function still returns back the original head in the list. I'm guessing this is because the reference for xs is still referring to the original xs list, problem is I can't override that because it's a val.
Any ideas what I'm doing wrong?
You should use the return value of the inner call to max and compare that to the local largest value.
Something like the following (removed println just for readability):
def max(xs: List[Int]): Int = {
var largest = xs.head
if (!xs.tail.isEmpty) {
var remaining = List[Int]()
remaining = largest :: xs.tail
var next = max(remaining)
largest = if (largest > next) largest else next
}
return largest
}
Bye.
I have an answer to your question but first...
This is the most minimal recursive implementation of max I've ever been able to think up:
def max(xs: List[Int]): Option[Int] = xs match {
case Nil => None
case List(x: Int) => Some(x)
case x :: y :: rest => max( (if (x > y) x else y) :: rest )
}
(OK, my original version was ever so slightly more minimal but I wrote that in Scheme which doesn't have Option or type safety etc.) It doesn't need an accumulator or a local helper function because it compares the first two items on the list and discards the smaller, a process which - performed recursively - inevitably leaves you with a list of just one element which must be bigger than all the rest.
OK, why your original solution doesn't work... It's quite simple: you do nothing with the return value from the recursive call to max. All you had to do was change the line
max(remaining)
to
largest = max(remaining)
and your function would work. It wouldn't be the prettiest solution, but it would work. As it is, your code looks as if it assumes that changing the value of largest inside the recursive call will change it in the outside context from which it was called. But each new call to max creates a completely new version of largest which only exists inside that new iteration of the function. Your code then throws away the return value from max(remaining) and returns the original value of largest, which hasn't changed.
Another way to solve this would have been to use a local (inner) function after declaring var largest. That would have looked like this:
def max(xs: List[Int]): Int = {
var largest = xs.head
def loop(ys: List[Int]) {
if (!ys.isEmpty) {
var next = ys.head
largest = if (largest > next) largest else next
loop(ys.tail)
}
}
loop(xs.tail)
return largest
}
Generally, though, it is better to have recursive functions be entirely self-contained (that is, not to look at or change external variables but only at their input) and to return a meaningful value.
When writing a recursive solution of this kind, it often helps to think in reverse. Think first about what things are going to look like when you get to the end of the list. What is the exit condition? What will things look like and where will I find the value to return?
If you do this, then the case which you use to exit the recursive function (by returning a simple value rather than making another recursive call) is usually very simple. The other case matches just need to deal with a) invalid input and b) what to do if you are not yet at the end. a) is usually simple and b) can usually be broken down into just a few different situations, each with a simple thing to do before making another recursive call.
If you look at my solution, you'll see that the first case deals with invalid input, the second is my exit condition and the third is "what to do if we're not at the end".
In many other recursive solutions, Nil is the natural end of the recursion.
This is the point at which I (as always) recommend reading The Little Schemer. It teaches you recursion (and basic Scheme) at the same time (both of which are very good things to learn).
It has been pointed out that Scala has some powerful functions which can help you avoid recursion (or hide the messy details of it), but to use them well you really do need to understand how recursion works.
The following is a typical way to solve this sort of problem. It uses an inner tail-recursive function that includes an extra "accumulator" value, which in this case will hold the largest value found so far:
def max(xs: List[Int]): Int = {
def go(xs: List[Int], acc: Int): Int = xs match {
case Nil => acc // We've emptied the list, so just return the final result
case x :: rest => if (acc > x) go(rest, acc) else go(rest, x) // Keep going, with remaining list and updated largest-value-so-far
}
go(xs, Int.MinValue)
}
Nevermind I've resolved the issue...
I finally came up with:
def max(xs: List[Int]): Int = {
var largest = 0
var remaining = List[Int]()
if (!xs.isEmpty) {
largest = xs.head
if (!xs.tail.isEmpty) {
var next = xs.tail.head
largest = if (largest > next) largest else next
remaining = largest :: xs.tail.tail
}
}
if (!remaining.tail.isEmpty) max(remaining) else xs.head
}
Kinda glad we have loops - this is an excessively complicated solution and hard to get your head around in my opinion. I resolved the problem by making sure the recursive call was the last statement in the function either that or xs.head is returned as the result if there isn't a second member in the array.
The most concise but also clear version I have ever seen is this:
def max(xs: List[Int]): Int = {
def maxIter(a: Int, xs: List[Int]): Int = {
if (xs.isEmpty) a
else a max maxIter(xs.head, xs.tail)
}
maxIter(xs.head, xs.tail)
}
This has been adapted from the solutions to a homework on the Scala official Corusera course: https://github.com/purlin/Coursera-Scala/blob/master/src/example/Lists.scala
but here I use the rich operator max to return the largest of its two operands. This saves having to redefine this function within the def max block.
What about this?
def max(xs: List[Int]): Int = {
maxRecursive(xs, 0)
}
def maxRecursive(xs: List[Int], max: Int): Int = {
if(xs.head > max && ! xs.isEmpty)
maxRecursive(xs.tail, xs.head)
else
max
}
What about this one ?
def max(xs: List[Int]): Int = {
var largest = xs.head
if( !xs.tail.isEmpty ) {
if(xs.head < max(xs.tail)) largest = max(xs.tail)
}
largest
}
My answer is using recursion is,
def max(xs: List[Int]): Int =
xs match {
case Nil => throw new NoSuchElementException("empty list is not allowed")
case head :: Nil => head
case head :: tail =>
if (head >= tail.head)
if (tail.length > 1)
max(head :: tail.tail)
else
head
else
max(tail)
}
}
I'm trying to do some experiment with Scala. I'd like to repeat this experiment (randomized) until the expected result comes out and get that result. If I do this with either while or do-while loop, then I need to write (suppose 'body' represents the experiment and 'cond' indicates if it's expected):
do {
val result = body
} while(!cond(result))
It does not work, however, since the last condition cannot refer to local variables from the loop body. We need to modify this control abstraction a little bit like this:
def repeat[A](body: => A)(cond: A => Boolean): A = {
val result = body
if (cond(result)) result else repeat(body)(cond)
}
It works somehow but is not perfect for me since I need to call this method by passing two parameters, e.g.:
val result = repeat(body)(a => ...)
I'm wondering whether there is a more efficient and natural way to do this so that it looks more like a built-in structure:
val result = do { body } until (a => ...)
One excellent solution for body without a return value is found in this post: How Does One Make Scala Control Abstraction in Repeat Until?, the last one-liner answer. Its body part in that answer does not return a value, so the until can be a method of the new AnyRef object, but that trick does not apply here, since we want to return A rather than AnyRef. Is there any way to achieve this? Thanks.
You're mixing programming styles and getting in trouble because of it.
Your loop is only good for heating up your processor unless you do some sort of side effect within it.
do {
val result = bodyThatPrintsOrSomething
} until (!cond(result))
So, if you're going with side-effecting code, just put the condition into a var:
var result: Whatever = _
do {
result = bodyThatPrintsOrSomething
} until (!cond(result))
or the equivalent:
var result = bodyThatPrintsOrSomething
while (!cond(result)) result = bodyThatPrintsOrSomething
Alternatively, if you take a functional approach, you're going to have to return the result of the computation anyway. Then use something like:
Iterator.continually{ bodyThatGivesAResult }.takeWhile(cond)
(there is a known annoyance of Iterator not doing a great job at taking all the good ones plus the first bad one in a list).
Or you can use your repeat method, which is tail-recursive. If you don't trust that it is, check the bytecode (with javap -c), add the #annotation.tailrec annotation so the compiler will throw an error if it is not tail-recursive, or write it as a while loop using the var method:
def repeat[A](body: => A)(cond: A => Boolean): A = {
var a = body
while (cond(a)) { a = body }
a
}
With a minor modification you can turn your current approach in a kind of mini fluent API, which results in a syntax that is close to what you want:
class run[A](body: => A) {
def until(cond: A => Boolean): A = {
val result = body
if (cond(result)) result else until(cond)
}
}
object run {
def apply[A](body: => A) = new run(body)
}
Since do is a reserved word, we have to go with run. The result would now look like this:
run {
// body with a result type A
} until (a => ...)
Edit:
I just realized that I almost reinvented what was already proposed in the linked question. One possibility to extend that approach to return a type A instead of Unit would be:
def repeat[A](body: => A) = new {
def until(condition: A => Boolean): A = {
var a = body
while (!condition(a)) { a = body }
a
}
}
Just to document a derivative of the suggestions made earlier, I went with a tail-recursive implementation of repeat { ... } until(...) that also included a limit to the number of iterations:
def repeat[A](body: => A) = new {
def until(condition: A => Boolean, attempts: Int = 10): Option[A] = {
if (attempts <= 0) None
else {
val a = body
if (condition(a)) Some(a)
else until(condition, attempts - 1)
}
}
}
This allows the loop to bail out after attempts executions of the body:
scala> import java.util.Random
import java.util.Random
scala> val r = new Random()
r: java.util.Random = java.util.Random#cb51256
scala> repeat { r.nextInt(100) } until(_ > 90, 4)
res0: Option[Int] = Some(98)
scala> repeat { r.nextInt(100) } until(_ > 90, 4)
res1: Option[Int] = Some(98)
scala> repeat { r.nextInt(100) } until(_ > 90, 4)
res2: Option[Int] = None
scala> repeat { r.nextInt(100) } until(_ > 90, 4)
res3: Option[Int] = None
scala> repeat { r.nextInt(100) } until(_ > 90, 4)
res4: Option[Int] = Some(94)
Much like this question:
Functional code for looping with early exit
Say the code is
def findFirst[T](objects: List[T]):T = {
for (obj <- objects) {
if (expensiveFunc(obj) != null) return /*???*/ Some(obj)
}
None
}
How to yield a single element from a for loop like this in scala?
I do not want to use find, as proposed in the original question, i am curious about if and how it could be implemented using the for loop.
* UPDATE *
First, thanks for all the comments, but i guess i was not clear in the question. I am shooting for something like this:
val seven = for {
x <- 1 to 10
if x == 7
} return x
And that does not compile. The two errors are:
- return outside method definition
- method main has return statement; needs result type
I know find() would be better in this case, i am just learning and exploring the language. And in a more complex case with several iterators, i think finding with for can actually be usefull.
Thanks commenters, i'll start a bounty to make up for the bad posing of the question :)
If you want to use a for loop, which uses a nicer syntax than chained invocations of .find, .filter, etc., there is a neat trick. Instead of iterating over strict collections like list, iterate over lazy ones like iterators or streams. If you're starting with a strict collection, make it lazy with, e.g. .toIterator.
Let's see an example.
First let's define a "noisy" int, that will show us when it is invoked
def noisyInt(i : Int) = () => { println("Getting %d!".format(i)); i }
Now let's fill a list with some of these:
val l = List(1, 2, 3, 4).map(noisyInt)
We want to look for the first element which is even.
val r1 = for(e <- l; val v = e() ; if v % 2 == 0) yield v
The above line results in:
Getting 1!
Getting 2!
Getting 3!
Getting 4!
r1: List[Int] = List(2, 4)
...meaning that all elements were accessed. That makes sense, given that the resulting list contains all even numbers. Let's iterate over an iterator this time:
val r2 = (for(e <- l.toIterator; val v = e() ; if v % 2 == 0) yield v)
This results in:
Getting 1!
Getting 2!
r2: Iterator[Int] = non-empty iterator
Notice that the loop was executed only up to the point were it could figure out whether the result was an empty or non-empty iterator.
To get the first result, you can now simply call r2.next.
If you want a result of an Option type, use:
if(r2.hasNext) Some(r2.next) else None
Edit Your second example in this encoding is just:
val seven = (for {
x <- (1 to 10).toIterator
if x == 7
} yield x).next
...of course, you should be sure that there is always at least a solution if you're going to use .next. Alternatively, use headOption, defined for all Traversables, to get an Option[Int].
You can turn your list into a stream, so that any filters that the for-loop contains are only evaluated on-demand. However, yielding from the stream will always return a stream, and what you want is I suppose an option, so, as a final step you can check whether the resulting stream has at least one element, and return its head as a option. The headOption function does exactly that.
def findFirst[T](objects: List[T], expensiveFunc: T => Boolean): Option[T] =
(for (obj <- objects.toStream if expensiveFunc(obj)) yield obj).headOption
Why not do exactly what you sketched above, that is, return from the loop early? If you are interested in what Scala actually does under the hood, run your code with -print. Scala desugares the loop into a foreach and then uses an exception to leave the foreach prematurely.
So what you are trying to do is to break out a loop after your condition is satisfied. Answer here might be what you are looking for. How do I break out of a loop in Scala?.
Overall, for comprehension in Scala is translated into map, flatmap and filter operations. So it will not be possible to break out of these functions unless you throw an exception.
If you are wondering, this is how find is implemented in LineerSeqOptimized.scala; which List inherits
override /*IterableLike*/
def find(p: A => Boolean): Option[A] = {
var these = this
while (!these.isEmpty) {
if (p(these.head)) return Some(these.head)
these = these.tail
}
None
}
This is a horrible hack. But it would get you the result you wished for.
Idiomatically you'd use a Stream or View and just compute the parts you need.
def findFirst[T](objects: List[T]): T = {
def expensiveFunc(o : T) = // unclear what should be returned here
case class MissusedException(val data: T) extends Exception
try {
(for (obj <- objects) {
if (expensiveFunc(obj) != null) throw new MissusedException(obj)
})
objects.head // T must be returned from loop, dummy
} catch {
case MissusedException(obj) => obj
}
}
Why not something like
object Main {
def main(args: Array[String]): Unit = {
val seven = (for (
x <- 1 to 10
if x == 7
) yield x).headOption
}
}
Variable seven will be an Option holding Some(value) if value satisfies condition
I hope to help you.
I think ... no 'return' impl.
object TakeWhileLoop extends App {
println("first non-null: " + func(Seq(null, null, "x", "y", "z")))
def func[T](seq: Seq[T]): T = if (seq.isEmpty) null.asInstanceOf[T] else
seq(seq.takeWhile(_ == null).size)
}
object OptionLoop extends App {
println("first non-null: " + func(Seq(null, null, "x", "y", "z")))
def func[T](seq: Seq[T], index: Int = 0): T = if (seq.isEmpty) null.asInstanceOf[T] else
Option(seq(index)) getOrElse func(seq, index + 1)
}
object WhileLoop extends App {
println("first non-null: " + func(Seq(null, null, "x", "y", "z")))
def func[T](seq: Seq[T]): T = if (seq.isEmpty) null.asInstanceOf[T] else {
var i = 0
def obj = seq(i)
while (obj == null)
i += 1
obj
}
}
objects iterator filter { obj => (expensiveFunc(obj) != null } next
The trick is to get some lazy evaluated view on the colelction, either an iterator or a Stream, or objects.view. The filter will only execute as far as needed.