Total newbie question here...Today while trying to calculate sum of a list of integers(actually BitSet), I ran into overflow scenarios and noticed that the return type of(sum/product) is Int. Are there any methods in Range/List to sum up or say multiply all values to Long?
val x = 1 to Integer.MaxValue
println(x.sum) //prints -1453759936
thanks
Convert the elements to Long (or BigInt should that go that far) while summing:
x.view.map(_.toLong).sum
You can also go back to fold
x.foldLeft(0L)(_ + _)
(Note: should you sum over a range, maybe it would be better do a little math, but I understand that is not what you did in fact)
Compare:
>> val x = 1 to Int.MaxValue
x: scala.collection.immutable.Range.Inclusive with scala.collection.immutable.Range.ByOne = Range(...)
With:
>> val x = 1L to Int.MaxValue
x: scala.collection.immutable.NumericRange.Inclusive[Long] = NumericRange(...)
Note that the first uses Int.to, and the latter used Long.to (where Int.MaxValue is up-converted automatically). Of course, the sum of a consecutive integer sequence has a very nice discrete formula :)
Happy coding.
This isn't very efficient, but the easiest way:
val x = 1L to Int.MaxValue
println(x.sum) //prints 2305843008139952128
If you need x to contain Ints rather than Longs, you can do
val x = 1 to Int.MaxValue
println(x.foldLeft(0L)(_+_))
Range.Long(1, Int.MaxValue, 1).sum
Related
I'm learning functional programming partially by reading the book Functional Programming in Scala a.k.a. The Red Book and I've run into my first real blocker. In Chapter 6, The book uses the example of a random number generator to illustrate how to change state by using side effects. Then the book goes on to demonstrate the patterns you would typically encounter as well as some of the tangents you might take in making a functional stateful api. My problem comes when trying to understand the following code:
type Rand[+A] = RNG => (A, RNG)
def map[A, B](s: Rand[A])(f: A => B): Rand[B] =
rng => {
val (nxt, nxtrng) = s(rng)
(f(nxt), nxtrng)
}
def nonNegativeLessThan(n: Int): Rand[Int] =
map(nonNegativeIntv2) { i =>
val mod = i % n
if (i + (n - 1) - mod >= 0) mod else nonNegativeLessThan(n)(???)
}
I hope this is enough context to get an idea what the code does. This is coming directly from the book on page 86. How does the if statement for method nonNegativeLessThan filters out values of i that are greater than the largest multiple of n in a 32 bit integer? Doesn't the recursive call in the else clause return a value of type Rand[Int]? In general I'm confused about what's going on in the code that is bolded. This is a pretty good book so I'm happy with how things are going so far and I feel I've learned a lot about functional programming. My apologies if this question is ill formed and if there is an issue with the formatting. This is my first post on stack overflow! Thank you to those who take a look at this and I hope it helps someone who has the same problem.
How does the if statement for method nonNegativeLessThan filters out values of i that are greater than the largest multiple of n in a 32 bit integer?
If i is greater than the largest multiple of n, then i + (n - 1) - mod will overflow and yield a negative number. The subsequent >= 0 is then false.
Doesn't the recursive call in the else clause return a value of type Rand[Int]?
Well, nonNegativeLessThan(n) is indeed of type Rand[Int]. However it says nonNegativeLessThan(n)(???), that is, it applies nonNegativeLessThan to n, and then it applies the resulting value (of type Rand[Int], which is a function type) to ???, and that yields an Int. Or rather it would do that if ??? ever yielded real value, which it doesn't.
The problem here is that you would have to pass the state of the RNG instead of ???, but the map function doesn't let you access that state. You'll need flatMap to solve this – presumably that's what the book is going to discuss next.
Imagine that you have a new domain of a set of integers, for example 0 to 11. The Rand type represents an aleatory number in that domain, but this type can not be skewed. That means that there can not be numbers that have a greater probability of being generated that others.
If you want to generate a number that is positive and less than four, you can use the map function of the Rand type, that allows to transform the result of a state action. First, it generates a nonnegative number and transform the result, via the map, applying the anonymous function: _ % n to obtain a number less than n.
def nonNegativeLessThan(n: Int): Rand[Int] =
map(nonNegativeInt) { _ % n }
It uses the modulo operation:
scala> 1 % 4
res0: Int = 1
scala> 2 % 4
res1: Int = 2
scala> 3 % 4
res2: Int = 3
scala> 4 % 4
res3: Int = 0
scala> 5 % 4
res4: Int = 1
scala> 6 % 4
res5: Int = 2
scala> 7 % 4
res6: Int = 3
scala> 8 % 4
res7: Int = 0
scala> 9 % 4
res8: Int = 1
scala> 10 % 4
res9: Int = 2
As you can see, the largest multiple of 4 in this domain is 8, so if the non-negative number that is generated is 9 or 10, we have a problem. The probability of having a 1 or 2 is greater than having a 3 or a 0. And for that reason, the other implementation detects that the number which is first generated as a result of the nonnegativeInt is not major than the largest multiple of n in a specific domain, the Int32 numbers domain in the book, to have a non- biased generator.
And yes, that book is amazing.
Hi I am new to scala functional programming methodology. I want to input a number to my function and check if it is a good number or not.
A number is a good number if its every digit is larger than the sum of digits which are on the right side of that digit.
For example:
9620 is good as (2 > 0, 6 > 2+0, 9 > 6+2+0)
steps I am using to solve this is
1. converting a number to string and reversing it
2. storing all digits of the reversed number as elements of a list
3. applying for loop from i equals 1 to length of number - 1
4. calculating sum of first i digits as num2
5. extracting ith digit from the list as digit1 which is one digit ahead of the first i numbers for which we calculated sum because list starts from zero.
6. comparing output of 4th and 5th step. if num1 is greater than num2 then we will break the for loop and come out of the loop to print it is not a good number.
please find my code below
val num1 = 9521.toString.reverse
val list1 = num1.map(_.todigit).toList
for (i <- 1 to num1.length - 1) {
val num2 = num1.take(i).map(_.toDigits) sum
val digit1 = list1(i)
if (num2 > digit1) {
print("number is not a good number")
break
}
}
I know this is not the most optimized way to solve this problem. Also I am looking for a way to code this using tail recursion where I pass two numbers and get all the good numbers falling in between those two numbers.
Can this be done in more optimized way?
Thanks in advance!
No String conversions required.
val n = 9620
val isGood = Stream.iterate(n)(_/10)
.takeWhile(_>0)
.map(_%10)
.foldLeft((true,-1)){ case ((bool,sum),digit) =>
(bool && digit > sum, sum+digit)
}._1
Here is a purely numeric version using a recursive function.
def isGood(n: Int): Boolean = {
#tailrec
def loop(n: Int, sum: Int): Boolean =
(n == 0) || (n%10 > sum && loop(n/10, sum + n%10))
loop(n/10, n%10)
}
This should compile into an efficient loop.
Using this function:(This will be the efficient way as the function forall will not traverse the entire list of digits. it stops when it finds the false condition immediately ( ie., when v(i)>v.drop(i+1).sum becomes false) while traversing from left to right of the vector v. )
def isGood(n: Int)= {
val v1 = n.toString.map(_.asDigit)
val v = if(v1.last!=0) v1 else v1.dropRight(1)
(0 to v.size-1).forall(i=>v(i)>v.drop(i+1).sum)
}
If we want to find good numbers in an interval of integers ranging from n1 to n2 we can use this function:
def goodNums(n1:Int,n2:Int) = (n1 to n2).filter(isGood(_))
In Scala REPL:
scala> isGood(9620)
res51: Boolean = true
scala> isGood(9600)
res52: Boolean = false
scala> isGood(9641)
res53: Boolean = false
scala> isGood(9521)
res54: Boolean = true
scala> goodNums(412,534)
res66: scala.collection.immutable.IndexedSeq[Int] = Vector(420, 421, 430, 510, 520, 521, 530, 531)
scala> goodNums(3412,5334)
res67: scala.collection.immutable.IndexedSeq[Int] = Vector(4210, 5210, 5310)
This is a more functional way. pairs is a list of tuples between a digit and the sum of the following digits. It is easy to create these tuples with drop, take and slice (a combination of drop and take) methods.
Finally I can represent my condition in an expressive way with forall method.
val n = 9620
val str = n.toString
val pairs = for { x <- 1 until str.length } yield (str.slice(x - 1, x).toInt, str.drop(x).map(_.asDigit).sum)
pairs.forall { case (a, b) => a > b }
If you want to be functional and expressive avoid to use break. If you need to check a condition for each element is a good idea to move your problem to collections, so you can use forAll.
This is not the case, but if you want performance (if you don't want to create an entire pairs collection because the condition for the first element is false) you can change your for collection from a Range to Stream.
(1 until str.length).toStream
Functional style tends to prefer monadic type things, such as maps and reduces. To make this look functional and clear, I'd do something like:
def isGood(value: Int) =
value.toString.reverse.map(digit=>Some(digit.asDigit)).
reduceLeft[Option[Int]]
{
case(sum, Some(digit)) => sum.collectFirst{case sum if sum < digit => sum+digit}
}.isDefined
Instead of using tail recursion to calculate this for ranges, just generate the range and then filter over it:
def goodInRange(low: Int, high: Int) = (low to high).filter(isGood(_))
I have written this function in Scala to calculate the fibonacci number given a particular index n:
def fibonacci(n: Long): Long = {
if(n <= 1) n
else
fibonacci(n - 1) + fibonacci(n - 2)
}
However it is not efficient when calculating with large indexes. Therefore I need to implement a function using a tuple and this function should return two consecutive values as the result.
Can somebody give me any hints about this? I have never used Scala before. Thanks!
This question should maybe go to Mathematics.
There is an explicit formula for the Fibonacci sequence. If you need to calculate the Fibonacci number for n without the previous ones, this is much faster. You find it here (Binet's formula): http://en.wikipedia.org/wiki/Fibonacci_number
Here's a simple tail-recursive solution:
def fibonacci(n: Long): Long = {
def fib(i: Long, x: Long, y: Long): Long = {
if (i > 0) fib(i-1, x+y, x)
else x
}
fib(n, 0, 1)
}
The solution you posted takes exponential time since it creates two recursive invocation trees (fibonacci(n - 1) and fibonacci(n - 2)) at each step. By simply tracking the last two numbers, you can recursively compute the answer without any repeated computation.
Can you explain the middle part, why (i-1, x+y, x) etc. Sorry if I am asking too much but I hate to copy and paste code without knowing how it works.
It's pretty simple—but my poor choice of variable names might have made it confusing.
i is simply a counter saying how many steps we have left. If we're calculating the Mth (I'm using M since I already used n in my code) Fibonacci number, then i tells us how many more terms we have left to calculate before we reach the Mth term.
x is the mth term in the Fibonacci sequence, or Fm (where m = M - i).
y is the m-1th term in the Fibonacci sequence, or Fm-1 .
So, on the first call fib(n, 0, 1), we have i=M, x=0, y=1. If you look up the bidirectional Fibonacci sequence, you'll see that F0 = 0 and F-1 = 1, which is why x=0 and y=1 here.
On the next recursive call, fib(i-1, x+y, x), we pass x+y as our next x value. This come straight from the definiton:
Fn = Fn-1 + Fn-2
We pass x as the next y term, since our current Fn-1 is the same as Fn-2 for the next term.
On each step we decrement i since we're one step closer to the final answer.
I am assuming that you don't have saved values from previous computations. If so, it will be faster for you to use the direct formula using the golden ratio instead of the recursive definition. The formula can be found in the Wikipedia page for Fibonnaci number:
floor(pow(phi, n)/root_of_5 + 0.5)
where phi = (1 + sqrt(5)/2).
I have no knowledge of programming in Scala. I am hoping someone on SO will upgrade my pseudo-code to actual Scala code.
Update
Here's another solution again using Streams as below (getting Memoization for free) but a bit more intuitive (aka: without using zip/tail invocation on fibs Stream):
val fibs = Stream.iterate( (0,1) ) { case (a,b)=>(b,a+b) }.map(_._1)
that yields the same output as below for:
fibs take 5 foreach println
Scala supports Memoizations through Streams that is an implementation of lazy lists. This is a perfect fit for Fibonacci implementation which is actually provided as an example in the Scala Api for Streams. Quoting here:
import scala.math.BigInt
object Main extends App {
val fibs: Stream[BigInt] = BigInt(0) #:: BigInt(1) #:: fibs.zip(fibs.tail).map { n => n._1 + n._2 }
fibs take 5 foreach println
}
// prints
//
// 0
// 1
// 1
// 2
// 3
I have recently been playing around on HackerRank in my down time, and am having some trouble solving this problem: https://www.hackerrank.com/challenges/functional-programming-the-sums-of-powers efficiently.
Problem statement: Given two integers X and N, find the number of ways to express X as a sum of powers of N of unique natural numbers.
Example: X = 10, N = 2
There is only one way get 10 using powers of 2 below 10, and that is 1^2 + 3^2
My Approach
I know that there probably exists a nice, elegant recurrence for this problem; but unfortunately I couldn't find one, so I started thinking about other approaches. What I decided on what that I would gather a range of numbers from [1,Z] where Z is the largest number less than X when raised to the power of N. So for the example above, I only consider [1,2,3] because 4^2 > 10 and therefore can't be a part of (positive) numbers that sum to 10. After gathering this range of numbers I raised them all to the power N then found the permutations of all subsets of this list. So for [1,2,3] I found [[1],[4],[9],[1,4],[1,9],[4,9],[1,4,9]], not a trivial series of operations for large initial ranges of numbers (my solution timed out on the final two hackerrank tests). The final step was to count the sublists that summed to X.
Solution
object Solution {
def numberOfWays(X : Int, N : Int) : Int = {
def candidates(num : Int) : List[List[Int]] = {
if( Math.pow(num, N).toInt > X )
List.range(1, num).map(
l => Math.pow(l, N).toInt
).toSet[Int].subsets.map(_.toList).toList
else
candidates(num+1)
}
candidates(1).count(l => l.sum == X)
}
def main(args: Array[String]) {
println(numberOfWays(readInt(),readInt()))
}
}
Has anyone encountered this problem before? If so, are there more elegant solutions?
After you build your list of squares you are left with what I would consider a kind of Partition Problem called the Subset Sum Problem. This is an old NP-Complete problem. So the answer to your first question is "Yes", and the answer to the second is given in the links.
This can be thought of as a dynamic programming problem. I still reason about Dynamic Programming problems imperatively, because that was how I was taught, but this can probably be made functional.
A. Make an array A of length X with type parameter Integer.
B. Iterate over i from 1 to Nth root of X. For all i, set A[i^N - 1] = 1.
C. Iterate over j from 0 until X. In an inner loop, iterate over k from 0 to (X + 1) / 2.
A[j] += A[k] * A[x - k]
D. A[X - 1]
This can be made slightly more efficient by keeping track of which indices are non-trivial, but not that much more efficient.
def numberOfWays(X: Int, N: Int): Int = {
def powerSumHelper(sum: Int, maximum: Int): Int = sum match {
case x if x < 1 => 0
case _ => {
val limit = scala.math.min(maximum, scala.math.floor(scala.math.pow(sum, 1.0 / N)).toInt)
(limit to 1 by -1).map(x => {
val y = scala.math.pow(x, N).toInt
if (y == sum) 1 else powerSumHelper(sum - y, x - 1)
}).sum
}
}
powerSumHelper(X, Integer.MAX_VALUE)
}
I have following simple code
def fib(i:Long,j:Long):Stream[Long] = i #:: fib(j, i+j)
(0l /: fib(1,1).take(10000000)) (_+_)
And it throws OutOfMemmoryError exception.
I can not understand why, because I think all the parts use constant memmory i.e. lazy evaluation streams and foldLeft...
Those code also don't work
fib(1,1).take(10000000).sum or max, min e.t.c.
How to correctly implement infinite streams and do iterative operations upon it?
Scala version: 2.9.0
Also scala javadoc said, that foldLeft operation is memmory safe for streams
/** Stream specialization of foldLeft which allows GC to collect
* along the way.
*/
#tailrec
override final def foldLeft[B](z: B)(op: (B, A) => B): B = {
if (this.isEmpty) z
else tail.foldLeft(op(z, head))(op)
}
EDIT:
Implementation with iterators still not useful, since it throws ${domainName} exception
def fib(i:Long,j:Long): Iterator[Long] = Iterator(i) ++ fib(j, i + j)
How to define correctly infinite stream/iterator in Scala?
EDIT2:
I don't care about int overflow, I just want to understand how to create infinite stream/iterator etc in scala without side effects .
The reason to use Stream instead of Iterator is so that you don't have to calculate all the small terms in the series over again. But this means that you need to store ten million stream nodes. These are pretty large, unfortunately, so that could be enough to overflow the default memory. The only realistic way to overcome this is to start with more memory (e.g. scala -J-Xmx2G). (Also, note that you're going to overflow Long by an enormous margin; the Fibonacci series increases pretty quickly.)
P.S. The iterator implementation I have in mind is completely different; you don't build it out of concatenated singleton Iterators:
def fib(i: Long, j: Long) = Iterator.iterate((i,j)){ case (a,b) => (b,a+b) }.map(_._1)
Now when you fold, past results can be discarded.
The OutOfMemoryError happens indenpendently from the fact that you use Stream. As Rex Kerr mentioned above, Stream -- unlike Iterator -- stores everything in memory. The difference with List is that the elements of Stream are calculated lazily, but once you reach 10000000, there will be 10000000 elements, just like List.
Try with new Array[Int](10000000), you will have the same problem.
To calculate the fibonacci number as above you may want to use different approach. You can take into account the fact that you only need to have two numbers, instead of the whole fibonacci numbers discovered so far.
For example:
scala> def fib(i:Long,j:Long): Iterator[Long] = Iterator(i) ++ fib(j, i + j)
fib: (i: Long,j: Long)Iterator[Long]
And to get, for example, the index of the first fibonacci number exceeding 1000000:
scala> fib(1, 1).indexWhere(_ > 1000000)
res12: Int = 30
Edit: I added the following lines to cope with the StackOverflow
If you really want to work with 1 millionth fibonacci number, the iterator definition above will not work either for StackOverflowError. The following is the best I have in mind at the moment:
class FibIterator extends Iterator[BigDecimal] {
var i: BigDecimal = 1
var j: BigDecimal = 1
def next = {val temp = i
i = i + j
j = temp
j }
def hasNext = true
}
scala> new FibIterator().take(1000000).foldLeft(0:BigDecimal)(_ + _)
res49: BigDecimal = 82742358764415552005488531917024390424162251704439978804028473661823057748584031
0652444660067860068576582339667553466723534958196114093963106431270812950808725232290398073106383520
9370070837993419439389400053162345760603732435980206131237515815087375786729469542122086546698588361
1918333940290120089979292470743729680266332315132001038214604422938050077278662240891771323175496710
6543809955073045938575199742538064756142664237279428808177636434609546136862690895665103636058513818
5599492335097606599062280930533577747023889877591518250849190138449610994983754112730003192861138966
1418736269315695488126272680440194742866966916767696600932919528743675517065891097024715258730309025
7920682881137637647091134870921415447854373518256370737719553266719856028732647721347048627996967...
#yura's problem:
def fib(i:Long,j:Long):Stream[Long] = i #:: fib(j, i+j)
(0l /: fib(1,1).take(10000000)) (_+_)
besides using a Long which can't possibly hold the Fibonacci of 10,000,000, it does work. That is, if the foldLeft is written as:
fib(1,1).take(10000000).foldLeft(0L)(_+_)
Looking at the Streams.scala source, foldLeft() is clearly designed for Garbage Collection, but /: is not def'd.
The other answers alluded to another problem. The Fibonacci of 10 million is a big number and if BigInt is used, instead of just overflowing like with a Long, absolutely enormous numbers are being added to each over and over again.
Since Stream.foldLeft is optimized for GC it does look like the way to solve for really big Fibonacci numbers, rather than using a zip or tail recursion.
// Fibonacci using BigInt
def fib(i:BigInt,j:BigInt):Stream[BigInt] = i #:: fib(j, i+j)
fib(1,0).take(10000000).foldLeft(BigInt("0"))(_+_)
Results of the above code: 10,000,000 is a 8-figure number. How many figures in fib(10000000)? 2,089,877
fib(1,1).take(10000000) is the "this" of the method /:, it is likely that the JVM will consider the reference alive as long as the method runs, even if in this case, it might get rid of it.
So you keep a reference on the head of the stream all along, hence on the whole stream as you build it to 10M elements.
You could just use recursion, which is about as simple:
def fibSum(terms: Int, i: Long = 1, j: Long = 1, total: Long = 2): Long = {
if (terms == 2) total
else fibSum(terms - 1, j, i + j, total + i + j)
}
With this, you can "fold" a billion elements in only a couple of seconds, but as Rex points out, summing the Fibbonaci sequence overflows Long very quickly.
If you really wanted to know the answer to your original problem and don't mind sacrificing some accuracy you could do this:
def fibSum(terms: Int, i: Double = 1, j: Double = 1, tot: Double = 2,
exp: Int = 0): String = {
if (terms == 2) "%.6f".format(tot) + " E+" + exp
else {
val (i1, j1, tot1, exp1) =
if (tot + i + j > 10) (i/10, j/10, tot/10, exp + 1)
else (i, j, tot, exp)
fibSum(terms - 1, j1, i1 + j1, tot1 + i1 + j1, exp1)
}
}
scala> fibSum(10000000)
res54: String = 2.957945 E+2089876