J. Abrahamson provided an in-depth answer to my composing-monads-v-applicative-functors question.
I gained some intuition, but I do not fully understand his helpful answer.
Given the following Either's:
scala> x
res0: Either[String,Int] = Right(100)
scala> err
res1: Either[Boolean,Int] = Left(false)
I tried to chain them together:
scala> for {
| xx <- x.right
| yy <- err.right
| } yield xx + yy
res3: scala.util.Either[Any,Int] = Left(false)
But, of course I don't want an Either[Any, Int]. Yet we get Any since, as I understand, the parent of Either[String, Int] and Either[Boolean, Int] is Either[Any, Int].
When building a for-comprehension, is the typical approach to find an end-type, i.e. Either[String, Int], and then make each flatMap call have that type?
If you mean that you want Either[Boolean, Int] instead of Either[Any, Int], then "You can't always get what you want". Composition of same (as in your example) Either types (but some other values) may return String instead of Boolean even for right projection:
scala> val x: Either[String,Int] = Left("aaa")
x: Either[String,Int] = Left(aaa)
scala> val r: Either[Boolean,Int] = Right(100)
r: Either[Boolean,Int] = Right(100)
scala> for {
| xx <- x.right
| yy <- r.right //you may swap xx and yy - it doesn't matter for this example, which means that flatMap could be called on any of it
| } yield yy + xx
res21: scala.util.Either[Any,Int] = Left(aaa)
So, Either[Any,Int] is really correct and smaller as possible end-type for it.
The desugared version:
scala> x.right.flatMap(xx => r.right.map(_ + xx))
res27: scala.util.Either[Any,Int] = Left(aaa)
Monad's flatMap signature:
flatMap[AA >: A, Y](f: (B) ⇒ Either[AA, Y]): Either[AA, Y]
Passed types:
flatMap[Any >: String, Int](f: (Int) ⇒ Either[?Boolean?, Int]): Either[Any, String]
AA >: A is completely legal here as it allows f = r.right.map(_ + xx) return left type bigger than String (so ?Boolean? becomes Any), otherwise it wouldn't even work. Answering your question, flatMap can't have AA = Boolean here, because A is already at least String, and as shown in first example can actually be a String.
And, by the way, there is no monads composition in this example - r could be just a functor. More than that, you're composing RightProjection with RightProjection here, so they commute automatically.
The only way to inferr Boolean is to kill String type with Nothing - you can do that only if you sure that x is always Right:
scala> val x: Either[Nothing,Int] = Right(100)
x: Either[Nothing,Int] = Right(100)
scala> val r: Either[Boolean,Int] = Left(false)
r: Either[Boolean,Int] = Left(false)
scala> for {
| yy <- r.right
| xx <- x.right
|
| } yield yy + xx
res24: scala.util.Either[Boolean,Int] = Left(false)
Then, of course you can't put strings, which is totally correct.
Related
I'm wondering how the type of the Stream below is casted?
type NTy = BigInt
def streamFib(n: Int): NTy = {
lazy val fibs: Stream[NTy] = 1 #:: 1 #:: fibs.zip(fibs.tail).map(n => n._1 + n._2)
fibs.drop(n - 1).head
}
This compiles and streamFib returns a BigInt type (as expected?).
I also noticed a few more behaviors:
val s1:Stream[BigInt] = 1 #:: 2 #:: Stream.empty // incorrect, found Int required BigInt
val s2:Stream[BigInt] = 1 #:: 2 #:: Stream.empty[BigInt] // correct
val l1:List[BigInt] = 1 :: 2 :: List.empty[BigInt] // incorrect, found Any required BigInt
val l2:List[BigInt] = 1 :: BigInt(2) :: List.empty[BigInt] // correct
tl;dr:
fibs is defined as a Stream[BigInt], so when you prepend an Int to it (1 #:: ...), the compiler looks for an implicit conversion from Int to BigInt and finds it in BigInt.int2bigInt
There are a couple of things going on here.
1) The BigInt companion object defines an implicit conversion from Int to BigInt:
implicit def int2bigInt(i: Int): BigInt = apply(i)
This means that wherever you need a BigInt you can supply an Int and the implicit conversion will convert the value. You can also say that Ints can be viewed as BigInts.
2) methods that end with a colon are right-associative. This means that 1 #:: 2 #:: Stream.empty[BigInt] can be effectively rewritten as Stream.empty[BigInt].#::(2).#::(1)
Now, if you look at the signature of Stream.#:: (def #::(hd: A): Stream[A]) you'll see that Stream[BigInt].#::(x) can only compile if x is a BigInt.
When you call 1 #:: 2 #:: Stream.empty[BigInt] you are calling Stream[BigInt].#:: passing an Int value instead of a BigInt, but, as I mentioned earlier, Ints can be viewed as BigInts, so they are automatically converted to BigInts and everything compiles.
When you do this: val s1:Stream[BigInt] = 1 #:: 2 #:: Stream.empty
you are doing a different thing instead: you are creating on the right hand side a Stream[Int] (Stream.empty type is inferred to be Int from the 1,2 values you pass) and then you are assigning this value to a Stream[BigInt] val.
Unfortunately, there is no implicit conversion from Stream[A] to Stream[B], even if A can be viewed as B, thus compilation fails.
You can define your own implicit conversion though:
implicit def asBigIntStream(xs: Stream[Int]): Stream[BigInt] = xs.map(BigInt.int2bigInt)
val s1:Stream[BigInt] = 1 #:: 2 #:: Stream.empty //This now works
There's something else going on with Lists: differently from Stream, the List cons is defined as:
def ::[B >: A] (x: B): List[B]
With Stream.#::(x) you needed x to be the exact same type as the Stream you were prepending x to. With List.::(x), instead, x (that has type B) can be an instance of a supertype of the list's type. The resulting list will be a List[B], i.e. prepending to a list can widen its type.
So, when you do 2 :: List.empty[BigInt] you're effectively Calling List[A].::(x: B) where A is BigInt and B is inferred to be Any because Any is the most strict supertype of BigInt that is also a supertype of Int.
Since this makes the compiler happy, no implicit conversions are looked for, and the resulting list is a List[Any] that you can't use anymore as a list of integers.
You can basically call whatever :: List[X] to get a List[Y] where Y is the most strict supertype of both X and the type of whatever
So, why doesn't val l1:List[BigInt] = 1 :: 2 :: List.empty[BigInt] work while val l2 : List[BigInt] = 1 :: BigInt(2) :: List.empty[BigInt] does?
It's because of type inference. Let's rewrite the two expressions removing the right-associativity:
val l1: List[BigInt] = (List.empty[BigInt].::(2)).::(1) // incorrect, found Any required BigInt
val l2: List[BigInt] = (List.empty[BigInt].::(BigInt(2))).::(1) // correct
I'm not 100% sure of this (anyone please correct me if I'm wrong):
The compiler can help the type inference only on the last application of ::
(List.empty[BigInt].::(2)) is a List[Any] well before applying .::(1) so there's nothing we can do
(List.empty[BigInt].::(BigInt(2))) is already a List[BigInt] and the compiler can try to make .::(1) a BigInt (thus looking for implicit conversions)
I am trying to solve a "simple" exercise in Scala. I have this function:
def mean(xs: Seq[Double]): Option[Double] =
if (xs.isEmpty) None
else Some(xs.sum / xs.length)
This is the text of the exercise:
EXERCISE 2: Implement the variance function (if the mean is m ,
variance is the mean of math.pow(x - m, 2) , see definition ) in terms
of mean and flatMap .
I was thinking something like
val xs = List(1.3,2.1,3.2)
val m = mean(xs)
xs flatMap(x=> math.pow(x-m,2)).mean //error!
What is the right way to solve? If possible I would like also a small theoretical explanation
I wouldn't call this simple.
map on Option[A] takes a function A => B and returns an Option[B] which is a Some if the original was a Some, and None otherwise. So Some(2).map(_+2) gives Some(4), but None[Int].map(_+2) gives None.
flatMap is much the same, but it takes a function A => Option[B] and still returns an Option[B]. If the original is None or the given function results in None, the result is None otherwise it's a Some[B].
You don't want to flatMap over the list, you want to flatMap the option. I'll give you a hint, it starts with:
mean(xs).flatMap(m => ...
After that arrow, m is a Double, not an Option[Double].
Edit:
Alright, fine, since we're getting all downvotey and not being helpful, here's the full answer:
def variance(xs: Seq[Double]): Option[Double] = {
mean(xs).flatMap(m => mean(xs.map(x => Math.pow(x-m, 2))))
}
Matt Putnam's answer gives some good pointers. I found it easier to understand this problem by breaking it up into smaller chunks.
scala> val xs = List(1.3,2.1,3.2)
xs: List[Double] = List(1.3, 2.1, 3.2)
scala> def mean(xs: Seq[Double]): Option[Double] = {
| if (xs.isEmpty) None
| else Some(xs.sum / xs.length)
| }
mean: (xs: Seq[Double])Option[Double]
Calculate the deviation of each element from the mean of the sequence:
scala> mean(xs).map(m => xs.map(x => x-m))
res0: Option[List[Double]] = Some(List(-0.9000000000000001, -0.10000000000000009, 1.0))
map works because if mean(xs) is Some, x => x-m is evaluated. Otherwise it isn't and we get None.
Now calculate the squared deviation from the mean of the sequence:
scala> mean(xs).map(m => xs.map(x => math.pow(x-m, 2)))
res1: Option[List[Double]] = Some(List(0.8100000000000003, 0.010000000000000018, 1.0))
Then calculate the mean of that:
scala> mean(xs).map(m => mean(xs.map(x => math.pow(x-m, 2))))
res2: Option[Option[Double]] = Some(Some(0.6066666666666668))
But here we have an Option that contains an Option, so we use flatMap to get the value of the first Some. So the solution with the desired return type Option[Double] is:
scala> mean(xs).flatMap(m => mean(xs.map(x => math.pow(x-m, 2))))
res3: Option[Double] = Some(0.6066666666666668)
I am taking the Functional Programming in Scala course on Coursera and I am having a hard time understanding this code snippet -
def sqrtStream(x: Double): Stream[Double] = {
def improve(guess: Double): Double = (guess+ x/ guess) / 2
lazy val guesses: Stream[Double] = 1 #:: (guesses map improve)
guesses
}
This method would find 10 approximate square root of 4 in increasing order of accuracy when I would do sqrtSteam(4).take(10).toList.
Can someone explain the evaluation strategy of guesses here? My doubt is what value of guesses in substituted when the second value of guesses is picked up?
Let's start from simplified example:
scala> lazy val a: Int = a + 5
a: Int = <lazy>
scala> a
stack overflow here, because of infinite recursion
So a is recalculating til it gets some stable value, like here:
scala> def f(f:() => Any) = 0 //takes function with captured a - returns constant 0
f: (f: () => Any)Int
scala> lazy val a: Int = f(() => a) + 5
a: Int = <lazy>
scala> a
res4: Int = 5 // 0 + 5
You may replace def f(f:() => Any) = 0 with def f(f: => Any) = 0, so a definition will look like it's really passed to the f: lazy val a: Int = f(a) + 5.
Streams use same mechanism - guesses map improve will be passed as parameter called by name (and lambda linked to the lazy a will be saved inside Stream, but not calculated until tail is requested), so it's like lazy val guesses = #::(1, () => guesses map improve). When you call guessess.head - tail will not be evaluated; guesses.tail will lazily return Stream (improve(1), ?), guesses.tail.tail will be Stream(improve(improve(1)), ?) and so on.
The value of guesses is not substituted. A stream is like a list, but its elements are evaluated only when they are needed and then they stored, so next time you access them the evaluation will not be necessary. The reference to the stream itself does not change.
On top of the example Αλεχει wrote, there is a nice explanation in Scala API:
http://www.scala-lang.org/api/current/index.html#scala.collection.immutable.Stream
You can easily find out what's going on by modifying the map function, as described in the scaladoc example:
scala> def sqrtStream(x: Double): Stream[Double] = {
| def improve(guess: Double): Double = (guess + x / guess) / 2
| lazy val guesses: Stream[Double] = 1 #:: (guesses map {n =>
| println(n, improve(n))
| improve(n)
| })
| guesses
| }
sqrtStream: (x: Double)Stream[Double]
The output is:
scala> sqrtStream(4).take(10).toList
(1.0,2.5)
(2.5,2.05)
(2.05,2.000609756097561)
(2.000609756097561,2.0000000929222947)
(2.0000000929222947,2.000000000000002)
(2.000000000000002,2.0)
(2.0,2.0)
(2.0,2.0)
(2.0,2.0)
res0: List[Double] = List(1.0, 2.5, 2.05, 2.000609756097561, 2.0000000929222947, 2.000000000000002, 2.0, 2.0, 2.0, 2.0)
I am reading "Programming in Scala 2nd Edition" and I have some idea about monad from a Haskell course I took. However, I do not understand why the following code "magically" works:
scala> val a: Option[Int] = Some(100)
a: Option[Int] = Some(100)
scala> val b = List(1, 2, 3)
b: List[Int] = List(1, 2, 3)
for ( y <- b; x <- a ) yield x;
res5: List[Int] = List(100, 100, 100)
I do not understand the above because according to the book's Chapter 23.4, the for expression is translated to something like:
b flatMap ( y =>
a map ( x => x )
)
I am puzzled why the above code compiles because y => a map (x => x) is of type Int => Option[Int], while the b.flatMap expects a Int => List[Something].
On the other hand, the following code does NOT compile (which is good otherwise I would be more lost):
scala> for ( x <- a; y <- b ) yield y;
<console>:10: error: type mismatch;
found : List[Int]
required: Option[?]
for ( x <- a; y <- b ) yield y;
^
So what is magical with the first example?
[…] b.flatMap expects a Int => List[Something].
That is not true: what it expects is an Int => GenTraversableOnce[Something]. (See http://www.scala-lang.org/api/current/index.html#scala.collection.immutable.List, and search the page for flatMap.) List[A] is a subtype of GenTraversableOnce[A] by inheritance. A function of type Int => List[Something] can be substituded because of the covariance of the result R of Function1, which is defined as: trait Function1[-T1, +R].
Option[A] is not a GenTraversableOnce[A], but there is an implicit conversion in Option's companion object: implicit def option2Iterable[A](xo: Option[A]): Iterable[A]. Iterable[A] is a subtype of GenTraversableOnce[A]. So the for-expression will get expanded to
b flatMap ( y =>
option2Iterable(a map ( x => x ))
)
On the other hand, the following code does NOT compile […]
This is because a.flatMap, by contrast, is more specific: it really does require an Int => Option[Something]. (See http://www.scala-lang.org/api/current/index.html#scala.Option, and search the page for flatMap.) This makes sense, since an Option[Something] can only hold one value, so you couldn't flatten an arbitrary GenTraversableOnce[Something] into it. The only thing that can successfully be flattened into an Option[Something] is another Option[Something].
I'd like to implement a "matrix dot product" in Scala in the following way:
type Real = Double
type Row = Array[Real]
type Matrix = Array[Row]
def dot[T](f: (T,T) => Real)(as: Iterable[T], bs: Iterable[T]): Real =
(for ((a, b) <- as zip bs) yield f(a, b)) sum
def rowDot(r1: Row, r2: Row) = dot(_*_)(r1, r2)
def matDot(m1: Matrix, m2: Matrix) = dot(rowDot)(m1, m2)
However, the definition of rowDot doesn't work. Scala needs explicit type annotations for the anonymous function (_*_), so instead I must write
def rowDot(r1: Row, r2: Row) = dot((x:Real, y: Real) => x*y)(r1, r2)
or
def rowDot = dot((x:Real, y: Real) => x*y) _
Is there some way to change the definition of dot so that the shorthand (_*_) can be used?
Edit: Another confusion: matDot also gives type errors in certain circumstances. It fails with Arrays of Arrays, but not with Lists of Arrays
scala> matDot(Array(Array(1.0,2.0)), Array(Array(1.0,2.0,3.0)))
<console>:27: error: type mismatch;
found : Array[Array[Double]]
required: Iterable[Iterable[Real]]
matDot(Array(Array(1.0,2.0)), Array(Array(1.0,2.0,3.0)))
^
scala> matDot(List(Array(1.0,2.0)), List(Array(1.0,2.0,3.0)))
res135: Real = 5.0
What's the difference?
specifying dot[Real] explicitly should work too.
def rowDot(r1: Row, r2: Row) = dot[Real](_*_)(r1, r2)
EDIT
replying to your edit: I think the issue is that the implicit conversion from Array to WrappedArray is not applied recursively when you have a Array[Array].
Array[Int] is not an Iterable[Int]; normally, when you assign it to a Iterable, an Array[Int] is implicitly converted to a WrappedArray[Int] (where WrappedArray is a Iterable[Int]). This is what happens when you use List[Array[Int]] (you get a List[WrappedArray[Int]] implicitly).
However, as I said, the implicit conversion is not applied recursively, so an Array[Array[Int]] is not implicitly converted to WrappedArray[WrappedArray[Int]].
Here's a REPL session that demonstrates the problem:
A List[Array[Int]] can be assigned to Iterable[Iterable[Int]] (note that Array is converted to WrappedArray)
scala> val i : Iterable[Iterable[Int]] = List(Array(1,2), Array(1,2,3))
i: Iterable[Iterable[Int]] = List(WrappedArray(1, 2), WrappedArray(1, 2, 3))
An Array[Array[Int]] does not work automatically (as you discovered)
scala> val j : Iterable[Iterable[Int]] = Array(Array(1,2), Array(1,2,3))
<console>:9: error: type mismatch;
found : Array[Array[Int]]
required: Iterable[Iterable[Int]]
val j : Iterable[Iterable[Int]] = Array(Array(1,2), Array(1,2,3))
^
However, with some hand-holding (converting manually the inner Arrays to WrappedArrays) everything works again:
scala> import scala.collection.mutable.WrappedArray
import scala.collection.mutable.WrappedArray
scala> val k : Iterable[Iterable[Int]] = Array(WrappedArray.make(Array(1,2)),
WrappedArray.make(Array(1,2,3)))
k: Iterable[Iterable[Int]] = WrappedArray(WrappedArray(1, 2), WrappedArray(1, 2,
3))
Yes - if you switch your argument lists around. Type inference on function parameters works more effectively when the function parameter is alone in the last argument list:
def dot[T](as: Iterable[T], bs: Iterable[T])(f: (T,T) => Real): Real =
(for ((a, b) <- as zip bs) yield f(a, b)) sum
def rowDot(r1: Row, r2: Row) = dot(r1, r2)(_*_)