How this stream's type is casted? - scala

I'm wondering how the type of the Stream below is casted?
type NTy = BigInt
def streamFib(n: Int): NTy = {
lazy val fibs: Stream[NTy] = 1 #:: 1 #:: fibs.zip(fibs.tail).map(n => n._1 + n._2)
fibs.drop(n - 1).head
}
This compiles and streamFib returns a BigInt type (as expected?).
I also noticed a few more behaviors:
val s1:Stream[BigInt] = 1 #:: 2 #:: Stream.empty // incorrect, found Int required BigInt
val s2:Stream[BigInt] = 1 #:: 2 #:: Stream.empty[BigInt] // correct
val l1:List[BigInt] = 1 :: 2 :: List.empty[BigInt] // incorrect, found Any required BigInt
val l2:List[BigInt] = 1 :: BigInt(2) :: List.empty[BigInt] // correct

tl;dr:
fibs is defined as a Stream[BigInt], so when you prepend an Int to it (1 #:: ...), the compiler looks for an implicit conversion from Int to BigInt and finds it in BigInt.int2bigInt
There are a couple of things going on here.
1) The BigInt companion object defines an implicit conversion from Int to BigInt:
implicit def int2bigInt(i: Int): BigInt = apply(i)
This means that wherever you need a BigInt you can supply an Int and the implicit conversion will convert the value. You can also say that Ints can be viewed as BigInts.
2) methods that end with a colon are right-associative. This means that 1 #:: 2 #:: Stream.empty[BigInt] can be effectively rewritten as Stream.empty[BigInt].#::(2).#::(1)
Now, if you look at the signature of Stream.#:: (def #::(hd: A): Stream[A]) you'll see that Stream[BigInt].#::(x) can only compile if x is a BigInt.
When you call 1 #:: 2 #:: Stream.empty[BigInt] you are calling Stream[BigInt].#:: passing an Int value instead of a BigInt, but, as I mentioned earlier, Ints can be viewed as BigInts, so they are automatically converted to BigInts and everything compiles.
When you do this: val s1:Stream[BigInt] = 1 #:: 2 #:: Stream.empty
you are doing a different thing instead: you are creating on the right hand side a Stream[Int] (Stream.empty type is inferred to be Int from the 1,2 values you pass) and then you are assigning this value to a Stream[BigInt] val.
Unfortunately, there is no implicit conversion from Stream[A] to Stream[B], even if A can be viewed as B, thus compilation fails.
You can define your own implicit conversion though:
implicit def asBigIntStream(xs: Stream[Int]): Stream[BigInt] = xs.map(BigInt.int2bigInt)
val s1:Stream[BigInt] = 1 #:: 2 #:: Stream.empty //This now works
There's something else going on with Lists: differently from Stream, the List cons is defined as:
def ::[B >: A] (x: B): List[B]
With Stream.#::(x) you needed x to be the exact same type as the Stream you were prepending x to. With List.::(x), instead, x (that has type B) can be an instance of a supertype of the list's type. The resulting list will be a List[B], i.e. prepending to a list can widen its type.
So, when you do 2 :: List.empty[BigInt] you're effectively Calling List[A].::(x: B) where A is BigInt and B is inferred to be Any because Any is the most strict supertype of BigInt that is also a supertype of Int.
Since this makes the compiler happy, no implicit conversions are looked for, and the resulting list is a List[Any] that you can't use anymore as a list of integers.
You can basically call whatever :: List[X] to get a List[Y] where Y is the most strict supertype of both X and the type of whatever
So, why doesn't val l1:List[BigInt] = 1 :: 2 :: List.empty[BigInt] work while val l2 : List[BigInt] = 1 :: BigInt(2) :: List.empty[BigInt] does?
It's because of type inference. Let's rewrite the two expressions removing the right-associativity:
val l1: List[BigInt] = (List.empty[BigInt].::(2)).::(1) // incorrect, found Any required BigInt
val l2: List[BigInt] = (List.empty[BigInt].::(BigInt(2))).::(1) // correct
I'm not 100% sure of this (anyone please correct me if I'm wrong):
The compiler can help the type inference only on the last application of ::
(List.empty[BigInt].::(2)) is a List[Any] well before applying .::(1) so there's nothing we can do
(List.empty[BigInt].::(BigInt(2))) is already a List[BigInt] and the compiler can try to make .::(1) a BigInt (thus looking for implicit conversions)

Related

Combining wrapped elements in Scala: Option[Array[Int]]

I'm trying to combine to arrays that are wrapped in an Option:
val a = Option(Array(1, 2, 3))
val b = Option(Array(4,5))
val q = for {
x <- a
y <- b
} yield x ++ y
The problem is that if b is None it returns None even though I'd like to have a. And if a is None the compiler complains that ++ is not a member of Nothing (even though I expect to receive b). Is this doable with the standard library or do I have to look at semigroups in Cats or Scalaz?
I tried the following in Cats but couldn't get it to work:
Semigroup[Option[Array[Int]]].combine(a,b) // === a |+| b
It tells me that:
could not find implicit value for parameter ev: cats.kernel.Semigroup[Option[Array[Int]]]
The resulting type should be the same as the types of a and b.
(a ++ b).flatten.toArray
The ++ method is not a part of the Option class, but it works here because of an implicit conversion. If you see the scala doc for Option, it says This member is added by an implicit conversion from Option[A] to Iterable[A] performed by method option2Iterable in scala.Option.
So, options can be treated as iterables.
Preserving the type Option[C[X]], where C is some collection type and X is the element type of that collection, I came up with:
a.fold(b)(x => b.fold(a)(y => Option(x ++ y)))
You should be able to do
val q = a.toList.flatten ++ b.toList.flatten

Correct way to work with two instances of Option together

When I have one Option[T] instance it is quite easy to perform any operation on T using monadic operations such as map() and flatMap(). This way I don't have to do checks to see whether it is defined or empty, and chain operations together to ultimately get an Option[R] for the result R.
My difficulty is whether there is a similar elegant way to perform functions on two Option[T] instances.
Lets take a simple example where I have two vals, x and y of type Option[Int]. And I want to get the maximum of them if they are both defined, or the one that is defined if only one is defined, and None if none are defined.
How would one write this elegantly without involving lots of isDefined checks inside the map() of the first Option?
You can use something like this:
def optMax(op1:Option[Int], op2: Option[Int]) = op1 ++ op2 match {
case Nil => None
case list => list.max
}
Or one much better:
def f(vars: Option[Int]*) = (for( vs <- vars) yield vs).max
#jwvh,thanks for a good improvement:
def f(vars: Option[Int]*) = vars.max
Usually, you'll want to do something if both values are defined.
In that case, you could use a for-comprehension:
val aOpt: Option[Int] = getIntOpt
val bOpt: Option[Int] = getIntOpt
val maxOpt: Option[Int] =
for {
a <- aOpt
b <- bOpt
} yield max(a, b)
Now, the problem you described is not as common. You want to do something if both values are defined, but you also want to retrieve the value of an option if only one of them is defined.
I would just use the for-comprehension above, and then chain two calls to orElse to provide alternative values if maxOpt turns out to be None.
maxOpt orElse aOpt orElse bOpt
orElse's signature:
def orElse[B >: A](alternative: ⇒ Option[B]): Option[B]
Here's another fwiw:
import scala.util.Try
def maxOpt (a:Option[Int]*)= Try(a.flatten.max).toOption
It works with n arguments (including zero arguments).
Pattern matching would allow something easy to grasp, but that might not be the most elegant way:
def maxOpt[T](optA: Option[T], optB: Option[T])(implicit f: (T, T) => T): Option[T] = (optA, optB) match {
case (Some(a), Some(b)) => Some(f(a, b))
case (None, Some(b)) => Some(b)
case (Some(a), None) => Some(a)
case (None, None) => None
}
You end up with something like:
scala> maxOpt(Some(1), None)(Math.max)
res2: Option[Int] = Some(1)
Once you have that building, block, you can use it inside for-comp or monadic operations.
To get maxOpt, you can also use an applicative, which using Scalaz would look like (aOpt |#| bOpt) { max(_, _) } & then chain orElses as #dcastro suggested.
I assume you expect Some[Int]|None as a result, not Int|None (otherwise return type has to be Any):
def maxOption(opts: Option[Int]*) = {
val flattened = opts.flatten
flattened.headOption.map { _ => flattened.max }
}
Actually, Scala already gives you this ability more or less directly.
scala> import Ordering.Implicits._
import Ordering.Implicits._
scala> val (a,b,n:Option[Int]) = (Option(4), Option(9), None)
a: Option[Int] = Some(4)
b: Option[Int] = Some(9)
n: Option[Int] = None
scala> a max b
res60: Option[Int] = Some(9)
scala> a max n
res61: Option[Int] = Some(4)
scala> n max b
res62: Option[Int] = Some(9)
scala> n max n
res63: Option[Int] = None
A Haskell-ish take on this question is to observe that the following operations:
max, min :: Ord a => a -> a -> a
max a b = if a < b then b else a
min a b = if a < b then a else b
...are associative:
max a (max b c) == max (max a b) c
min a (min b c) == min (min a b) c
As such, any type Ord a => a together with either of these operations is a semigroup, a concept for which reusable abstractions can be built.
And you're dealing with Maybe (Haskell for "option"), which adds a generic "neutral" element to the base a type (you want max Nothing x == x to hold as a law). This takes you into monoids, which are a subtype of semigroups.
The Haskell semigroups library provides a Semigroup type class and two wrapper types, Max and Min, that generically implement the corresponding behaviors.
Since we're dealing with Maybe, in terms of that library the type that captures the semantics you want is Option (Max a)—a monoid that has the same binary operation as the Max semigroup, and uses Nothing as the identity element. So then the function simply becomes:
maxOpt :: Ord a => Option (Max a) -> Option (Max a) -> Option (Max a)
maxOpt a b = a <> b
...which since it's just the <> operator for Option (Max a) is not worth writing. You also gain all the other utility functions and classes that work on Semigroup and Monoid, so for example to find the maximum element of a [Option (Max a)] you'd just use the mconcat function.
The scalaz library comes with a Semigroup and a Monoid trait, as well as Max, Min, MaxVal and MinVal tags that implement those traits, so in fact the stuff that I've demonstrated here in Haskell exists in scalaz as well.

Scala: Option[T] as ?[T] (or even T?)

i tried
type ?[_] = Option[_]
def f(x: ?[Int]) = for (y <- x) yield y
(but i don't know what i am doing.)
insofar as types are just objects, i should be able to define a postix operator (i.e. zero arity method) for use in type signatures (i think). it might need a space like
def f(x: Int ?) = for (y <- x) yield y
scala makes it easy to use the Option type with matching and polymorphism, avoid null. but, most classes are (nullable) vars and java often returns vars. using classes and calling java are two of scala's selling points. an easy-to-write and easy-to-read syntax would support Options even more strongly.
what are all the things that scala does with "?" that make its parsing special.
ideally, one could write
def f(x: Int?) = for (y <- x) yield y
like other languages. can we do this in scala (without a macro)?
First, types are not objects. In fact, Scala has exactly two namespaces: values and types. They are very different things, and play by very different rules.
The postfix idea is kind of nice, actually, but it is not possible. There's an infix notation for types, though.
Now, to what you wrote:
type ?[_] = Option[_]
Each underscore has a different meaning. The underscore in ?[_] means ? is higher-kinded, but you don't care what it's type parameter is. The underscore in Option[_] means Option is an existential type. So when you write x: ?[Int], Scala will convert it to x: Option[t] { forSome type t }. This means that not only you don't get the Int, but the type parameter of Option is unknowable (you just know it exists).
However, it does compile:
scala> def f(x: ?[Int]) = for (y <- x) yield y
f: (x: ?[Int])Option[Any]
Which version of Scala did you use? 2.11? A co-worker of mine has already found some type inference regressions on 2.11, so that could be it.
The proper way to write the type alias would be this:
type ?[+A] = Option[A]
Not only we pass the type parameter along (it is a parameter, after all!), but we need to specify co-variance for it to act just Option (which is co-variant itself).
Now, as to your two questions, Scala has absolutely no special treatment of ?. And, no, you can't do this. This ? is not exactly widespread among languages either, and in all of them that support it, it is built in the language, and not something externally defined.
Besides, it's kind of a joke that, when interface with Java, typing out Option would be a problem -- not with the average identifier size in Java!
You intended to get an Option[Int] out:
scala> type ?[A] = Option[A]
defined type alias $qmark
scala> def f(x: ?[Int]) = for (y <- x) yield y + 1
f: (x: ?[Int])Option[Int]
and it does compile anyway.
You could maybe
scala> type ?[A,_] = Option[A]
defined type alias $qmark
scala> def f(x: Int ? _) = for (y <- x) yield y + 1
f: (x: ?[Int, _])Option[Int]
or similar.
scala> def f(x: Int ?_) = for (y <- x) yield y + 1
f: (x: ?[Int, _])Option[Int]
looks more postfixy.
P.S. Still curious whether variance annotation on type alias is required or merely advisable.
scala> type ?[A] = Option[A]
defined type alias $qmark
scala> trait X ; trait Y extends X ; trait Z extends X
defined trait X
defined trait Y
defined trait Z
scala> val x: ?[X] = null.asInstanceOf[?[Y]] // why is this OK?
x: ?[X] = null
scala> class C[A]
defined class C
scala> val c: C[X] = null.asInstanceOf[C[Y]] // like this is not OK
<console>:10: error: type mismatch;
found : C[Y]
required: C[X]
Note: Y <: X, but class C is invariant in type A.
You may wish to define A as +A instead. (SLS 4.5)
val c: C[X] = null.asInstanceOf[C[Y]]
^
Maybe compare SI-8522 and related issues.
You might consider a renaming import. When you create a type alias you only rename a type. When you rename a symbol during import you include all referents of that name, both type and value.
To wit:
scala> import scala.{Option => ?}
import scala.{Option=>$qmark}
scala> val oi1: ?[Int] = Some(1)
oi1: Option[Int] = Some(1)
scala> def mi1(oi: ?[Int]): Int = oi.getOrElse(-1)
mi1: (oi: Option[Int])Int
scala> mi1(None)
res1: Int = -1
scala> mi1(?(1))
res2: Int = 1
Compare with this:
scala> type ?[A] = Option[A]
scala> def mi1(oi: ?[Int]): Int = oi.getOrElse(-1)
mi1: (oi: ?[Int])Int
scala> mi1(?(1))
<console>:10: error: not found: value ?
mi1(?(1))
^

Scala by Example - problems with currying

Not sure how to properly formulate the question, there is a problem with currying in the merge sort example from Scala by Example book on page 69. The function is defined as follows:
def msort[A](less: (A, A) => Boolean)(xs: List[A]): List[A] = {
def merge(xs1: List[A], xs2: List[A]): List[A] =
if (xs1.isEmpty) xs2
else if (xs2.isEmpty) xs1
else if (less(xs1.head, xs2.head)) xs1.head :: merge(xs1.tail, xs2)
else xs2.head :: merge(xs1, xs2.tail)
val n = xs.length/2
if (n == 0) xs
else merge(msort(less)(xs take n), msort(less)(xs drop n))
}
and then there is an example of how to create other functions from it by currying:
val intSort = msort((x : Int, y : Int) => x < y)
val reverseSort = msort((x:Int, y:Int) => x > y)
however these two lines give me errors about insufficient number of arguments. And if I do like this:
val intSort = msort((x : Int, y : Int) => x < y)(List(1, 2, 4))
val reverseSort = msort((x:Int, y:Int) => x > y)(List(4, 3, 2))
it WILL work. Why? Can someone explain? Looks like the book is really dated since it is not the first case of such an inconsistance in its examples. Could anyone point to something more real to read? (better a free e-book).
My compiler (2.9.1) agrees, there seems to be an error here, but the compiler does tell you what to do:
error: missing arguments for method msort in object $iw;
follow this method with `_' if you want to treat it as a partially applied function
So, this works:
val intSort = msort((x : Int, y : Int) => x < y) _
Since the type of intSort is inferred, the compiler doesn't otherwise seem to know whether you intended to partially apply, or whether you missed arguments.
The _ can be omitted when the compiler can infer from the expected type that a partially applied function is what is intended. So this works too:
val intSort: List[Int] => List[Int] = msort((x: Int, y: Int) => x < y)
That's obviously more verbose, but more often you will take advantage of this without any extra boilerplate, for example if msort((x: Int, y: Int) => x < y) were the argument to a function where the parameter type is already known to be List[Int] => List[Int].
Edit: Page 181 of the current edition of the Scala Language Specification mentions a tightening of rules for implicit conversions of partially applied methods to functions since Scala 2.0. There is an example of invalid code very similar to the one in Scala by Example, and it is described as "previously legal code".

Polymorphic dot product in Scala and anonymous function shorthand

I'd like to implement a "matrix dot product" in Scala in the following way:
type Real = Double
type Row = Array[Real]
type Matrix = Array[Row]
def dot[T](f: (T,T) => Real)(as: Iterable[T], bs: Iterable[T]): Real =
(for ((a, b) <- as zip bs) yield f(a, b)) sum
def rowDot(r1: Row, r2: Row) = dot(_*_)(r1, r2)
def matDot(m1: Matrix, m2: Matrix) = dot(rowDot)(m1, m2)
However, the definition of rowDot doesn't work. Scala needs explicit type annotations for the anonymous function (_*_), so instead I must write
def rowDot(r1: Row, r2: Row) = dot((x:Real, y: Real) => x*y)(r1, r2)
or
def rowDot = dot((x:Real, y: Real) => x*y) _
Is there some way to change the definition of dot so that the shorthand (_*_) can be used?
Edit: Another confusion: matDot also gives type errors in certain circumstances. It fails with Arrays of Arrays, but not with Lists of Arrays
scala> matDot(Array(Array(1.0,2.0)), Array(Array(1.0,2.0,3.0)))
<console>:27: error: type mismatch;
found : Array[Array[Double]]
required: Iterable[Iterable[Real]]
matDot(Array(Array(1.0,2.0)), Array(Array(1.0,2.0,3.0)))
^
scala> matDot(List(Array(1.0,2.0)), List(Array(1.0,2.0,3.0)))
res135: Real = 5.0
What's the difference?
specifying dot[Real] explicitly should work too.
def rowDot(r1: Row, r2: Row) = dot[Real](_*_)(r1, r2)
EDIT
replying to your edit: I think the issue is that the implicit conversion from Array to WrappedArray is not applied recursively when you have a Array[Array].
Array[Int] is not an Iterable[Int]; normally, when you assign it to a Iterable, an Array[Int] is implicitly converted to a WrappedArray[Int] (where WrappedArray is a Iterable[Int]). This is what happens when you use List[Array[Int]] (you get a List[WrappedArray[Int]] implicitly).
However, as I said, the implicit conversion is not applied recursively, so an Array[Array[Int]] is not implicitly converted to WrappedArray[WrappedArray[Int]].
Here's a REPL session that demonstrates the problem:
A List[Array[Int]] can be assigned to Iterable[Iterable[Int]] (note that Array is converted to WrappedArray)
scala> val i : Iterable[Iterable[Int]] = List(Array(1,2), Array(1,2,3))
i: Iterable[Iterable[Int]] = List(WrappedArray(1, 2), WrappedArray(1, 2, 3))
An Array[Array[Int]] does not work automatically (as you discovered)
scala> val j : Iterable[Iterable[Int]] = Array(Array(1,2), Array(1,2,3))
<console>:9: error: type mismatch;
found : Array[Array[Int]]
required: Iterable[Iterable[Int]]
val j : Iterable[Iterable[Int]] = Array(Array(1,2), Array(1,2,3))
^
However, with some hand-holding (converting manually the inner Arrays to WrappedArrays) everything works again:
scala> import scala.collection.mutable.WrappedArray
import scala.collection.mutable.WrappedArray
scala> val k : Iterable[Iterable[Int]] = Array(WrappedArray.make(Array(1,2)),
WrappedArray.make(Array(1,2,3)))
k: Iterable[Iterable[Int]] = WrappedArray(WrappedArray(1, 2), WrappedArray(1, 2,
3))
Yes - if you switch your argument lists around. Type inference on function parameters works more effectively when the function parameter is alone in the last argument list:
def dot[T](as: Iterable[T], bs: Iterable[T])(f: (T,T) => Real): Real =
(for ((a, b) <- as zip bs) yield f(a, b)) sum
def rowDot(r1: Row, r2: Row) = dot(r1, r2)(_*_)