I'm using Circe and noticed something that i am not so confortable with and would like to understand what is going on under the hood ?
Fundamentally it is not really a circe issue. Also i was just playing with circe around to test few thing. So could have decoded in JsonObject straight but that is beside the point.
val jobjectStr = """{
| "idProperty": 1991264,
| "nIndex": 0,
| "sPropertyValue": "0165-5728"
| }""".stripMargin
val jobject = decode[Json](jobjectStr).flatMap{ json =>
json.as[JsonObject]
}
My issue is with the flapMap signature of Either, contravariance and what is happening here:
We have the following types:
decode[Json](jobjectStr): Either[Error, Json]
json.as[JsonObject]: Decoder.Result[JsonObject]
where circe defines
final type Result[A] = Either[DecodingFailure, A]
and
sealed abstract class DecodingFailure(val message: String) extends Error {
Now the signature of flatMap in either is:
def flatMap[A1 >: A, B1](f: B => Either[A1, B1]): Either[A1, B1]
In other words, talking only about type it is like my code is doing
Either[Error, Json] flatMap Either[DecodingFailure, JsonObject]
Hence my issue is: DecodingFailure >: Error is not true
And Indeed the type of the full expression is:
decode[Json](jobjectStr).flatMap{ json =>
json.as[JsonObject]
}: Either[Error, JsonObject]
Hence i'm confused, because my understanding is that the type of the first Parameter of Either is Contravariant in the flatMap Signature. Here there seems to be some wierd least upper bound inferencing going on ... But i am not sure why or if it is even the case.
Any explanation ?
This really isn't a variance issue. A1 >: A is just telling us that the result type, A1, might have to be a super-type of the received type, A, if the compiler has to go looking for a least upper bound (the LUB). (The use of A1 in the f: B => ... description is, I think, a bit confusing.)
Consider the following:
class Base
class SubA extends Base
class SubB extends Base
Either.cond(true, "a string", new SubA)
.flatMap(Either.cond(true, _, new SubB))
//res0: scala.util.Either[Base,String] = Right(a string)
Notice how the result is Either[Base,String] because Base is the LUB of SubA and SubB.
So first of all, we need to understand that the compiler will always try to infer types that allow compilation. The only real way to avoid something to compile is to use implicits.
(not sure if this is part of the language specification, or a compiler implementation detail, or something common to all compilers, or a bug or a feature).
Now, let's start with a simpler example List and ::.
sealed trait List[+A] {
def ::[B >: A](b: B): List[B] = Cons(b, this)
}
final case class Cons[+A](head: A, tail: List[A]) extends List[A]
final case object Nil extends List[Nothing]
So, assuming the compiler will always allow some code like x :: list will always compile. Then, we have three scenarios:
x is of type A and list is a List[A], so it is obvious that the returned value has to be of type List[A].
x is of some type C and list is a List[A], and C is a subtype of A (C <: A). Then, the compiler simply upcast x to be of type A and the process continues as the previous one.
x is of some type D and list is a List[A], and D is not a subtype of A. Then, the compiler finds a new type B which is the LUB between D and A, the compiler finally upcast both x to be of type B and list to be a List[B] (this is possible due covariance) and proceeds like the first one.
Also, note that due to the existence of types like Any and Nothing there is "always" a LUB between two types.
Now let's see Either and flatMap.
sealed trait Either[+L, +R] {
def flatMap[LL >: L, RR](f: R => Either[LL, RR]): Either[LL, RR]
}
final case class Left[+L](l: L) extends Either[L, Nothing]
final case clas Right[+R](r: R) extends Either[Nothing, R]
Now, assuming my left side is an Error, I feel this behaviour of returning the LUB between the two possible lefts is the best, since at the end I would have the first error, or the second error or the final value, so since I do not know which of the two errors it was then that error must be of some type that encapsulates both possible errors.
Related
I don't understand why the following scala code doesn't compile:
sealed trait A
case class B() extends A {
def funcB: B = this
}
case class C() extends A {
def funcC: C = this
}
def f[T <: A](s:T): T = s match {
case s: B => s.funcB
case s: C => s.funcC
}
It works to replace f with
def f[T <: A](s:T): A = s match {
case s: B => s.funcB
case s: C => s.funcC
}
and then cast to the subtype when f is called, using asInstanceOf, for example. But I would like to be able to construct a function which unifies some previously defined methods, and have them be type stable. Can anyone please explain?
Also, note that the following f also compiles:
def f[T <: A](s:T): T = s match {
case s: B => s
case s: C => s
}
What makes it work?
In particular, in Scala 3 you could use match types
scala> type Foo[T <: A] = T match {
| case B => B
| case C => C
| }
|
| def f[T <: A](s:T): Foo[T] = s match {
| case s: B => s.funcB
| case s: C => s.funcC
| }
def f[T <: A](s: T): Foo[T]
scala> f(B())
val res0: B = B()
scala> f(C())
val res1: C = C()
In general, for the solution to "return current type" problems see Scala FAQ How can a method in a superclass return a value of the “current” type?
Compile-time techniques such as type classes and match types can be considered as kind of compile-time pattern matching which instruct the compiler to reduce to the most specific informationally rich type used at call site instead of otherwise having to determine a probably poorer upper bound type.
Why it does not work?
The key concept to understand is that parametric polymorphism is a kind of universal quantification which means it must make sense to the compiler for all instantiations of type parameters at call-sites. Consider typing specification
def f[T <: A](s: T): T
which the compiler might interpret something like so
For all types T that are a subtype of A, then f should return that
particular subtype T.
hence the expression expr representing the body of f
def f[T <: A](s:T): T = expr
must type to that particular T. Now lets try to type our expr
s match {
case s: B => s.funcB
case s: C => s.funcC
}
The type of
case s: B => s.funcB
is B, and the type of
case s: C => s.funcC
is C. Given we have B and C, now compiler has to take the least upper bound of the two which is A. But A is certainly not always T. Hence the typecheck fails.
Now lets do the same exercise with
def f[T <: A](s: T): A
This specification means (and observe the "for all" again)
For all types T that are a subtype of A, then f should return
their supertype A.
Now lets type the method body expressions
s match {
case s: B => s.funcB
case s: C => s.funcC
}
As before we arrive at types B and C, so compiler takes the upper bound which is the supertype A. And indeed this is the very return type we specified. So typecheck succeeds. However despite succeeding, at compile-time we lost some typing information as compiler will no longer consider all the information that comes with specific T passed in at call-site but only the information available via its supertype A. For example, if T has a member not existing in A, then we will not be able to call it.
What to avoid?
Regarding asInstanceOf, this is us telling the compiler to stop helping us because we will take the rains. Two groups of people tend to use it in Scala to make things work, the mad scientist library authors and ones transitioning from other more dynamically typed languages. However in most application level code it is considered bad practice.
It all comes down to our old friend (fiend?) the compile-time/run-time barrier. (And ne'er the twain shall meet.)
T is resolved at compile-time at the call site. When the compiler sees f(B) then T means B and when the compiler sees f(C) then T becomes C.
But match { case ... is resolved at run-time. The compiler can't know which case branch will be chosen. From the compiler's point of view all case options are equally likely. So if T is resolved to B but the code might take a C branch...well, the compiler can't allow that.
Looking at what does compile:
def f[T <: A](s:T): A = s match { //f() returns an A
case s: B => s.funcB //B is an A sub-type
case s: C => s.funcC //C is an A sub-type
} //OK, all is good
Your 2nd "also works" example does not compile for me.
To answer the question why it does not work.
f returns the result of the statement s match {...}.
The type of that statement is A (sometimes it returns B, and sometimes it returns C), not T as it is supposed to be. T is sometimes C, and sometimes B, s match {...} is never either of those. It is the supertype of them, which is A.
Re. this:
s match {
case s: B => s
case s: C => s
}
The type of this statement is obviously T, because s is T. It certainly does compile despite what #jwvh might be saying :)
In Scala's upper bound concept, the given type or its super type can be passed. For example, in the below method S is the type and A is the parameter we pass. This method accepts all the values present in Scala's type system actually. S, S subtype and its super type. This is due to the fact that all types extends Any type.
def method[A >: S](a:A) = { ... }
Then why can't we write all the upper bound notations as Any (which is the universal type in Scala). The above definition can be re-written as:
def met(a:Any) = { ... }
This is easy to understand.
What sort of advantage the upperbound brings in ?
Thanks!
It allows you to lose less type information than you would by going to Any.
For example If you have Dog and Cat inherit Animal, this works:
val maybeDog: Option[Dog] = ???
val pet = maybeDog.getOrElse(Cat())
Because getOrElse has a signature of def getOrElse[B >: A](default: => B): B, it is inferred that B is Animal (as the least upper bound of Cat and Dog), so the static type of val pet is Animal. If it was using Any, the result would require unsafe casting to work further. If it was not introducing a new type parameter altogether, you would be forced to write (maybeDog: Option[Animal]).getOrElse(Cat()) to achieve the same unification.
Additionally, it restricts the implementation to not do something completely silly. For example, this typechecks:
def getOrElse[A](option: Option[A])(default: => Any): Any = 42 // Int is Any, so why not?
While this doesn't:
def getOrElse[A, B >: A](option: Option[A])(default: => B): B = 42
Because while anything can go as B, that doesn't imply that Int is always a subtype of B.
I have a question that's been bugging me.
Lists in Scala are covariant (List[+A])
Let's say we have these classes:
class A
class B extends A
The map function of List[B] takes a function f: B => C
But I can also use a f: A => C
which is a subclass of f: B => C
and it totally makes sense.
What I am currently confused by is that
the map function should accept only functions that are superclasses of the original function (since functions are contravariant on their arguments), which does not apply in the example I've given.
I know there's something wrong with my logic and I would like to enlightened.
Your error lies in the assumption that map(f: A => C) should only accept functions that are superclasses of A => C.
While in reality, map will accept any function that is a subclass of A => C.
In Scala, a function parameter can always be a subclass of the required type.
The covariance of A in List[A] only tells you that, wherever a List[A] is required, you can provide a List[B], as long as B <: A.
Or, in simpler words: List[B] can be treated as if it was a subclass of List[A].
I have compiled a small example to explain these two behaviours:
class A
class B extends A
// this means: B <: A
val listA: List[A] = List()
val listB: List[B] = List()
// Example 1: List[B] <: List[A]
// Note: Here the List[+T] is our parameter! (Covariance!)
def printListA(list: List[A]): Unit = println(list)
printListA(listA)
printListA(listB)
// Example 2: Function[A, _] <: Function[B, _]
// Note: Here a Function1[-T, +R] is our parameter (Contravariance!)
class C
def fooA(a: A): C = ???
def fooB(b: B): C = ???
listB.map(fooB)
listB.map(fooA)
Try it out!
I hope this helps.
As you already suspected, you are mixing up things here.
On the one hand, you have a List[+A], which tells us something about the relationships between List[A] and List[B], given a relationship between A and B. The fact that List is covariant in A just means that List[B] <: List[A] when B <: A, as you know already know.
On the other hand, List exposes a method map to change its "contents". This method does not really care about List[A], but only about As, so the variance of List is irrelevant here. The bit that is confusing you here is that there is indeed some sub-typing to take into consideration: map accepts an argument (a A => C in this case, but it's not really relevant) and, as usual with methods and functions, you can always substitute its argument with anything that is a subtype of it. In your specific case, any AcceptedType will be ok, as long as AcceptedType <: Function1[A,C]. The variance that matters here is Function's, not List's.
I'm reading Functional Programming in Scala, and in chapter 04 the authors implement Option on their own. Now, when defining the function getOrElse they use an upper bound to restrict the type of A to a supertype (if a understood correctly)
So, the definition goes:
sealed trait Option[+A] {
def getOrElse[B >: A](default: => B): B = this match {
case None => default
case Some(a) => a
}
}
So, when we have something like
val a = Some(4)
println(a.getOrElse(None)) => println prints a integer value
val b = None
println(b.getOrElse(Some(3)) => println prints a Option[Integer] value
a has type Option[Int], so A would be type Int. B would be type Nothing. Nothing is a subtype of every other type. That means that Option[Nothing] is a subtype of Option[Int] (because of covariance), right?
But with B >: A we said that B has to be a supertype?! So how can we get an Int back? This is a bit confusing for me...
Anyone care to try and clarify?
That means that Option[Nothing] is a subtype of Option[Int] (because of covariance), right?
Correct. Option[Nothing] is an Option[Int].
But with B >: A we said that B has to be a supertype?! So how can we get an Int back?
It doesn't have to be a super-type. It just requires A as a lower-bound. Which means you can still pass Int to getOrElse if A is Int.
But that doesn't mean you can't pass instances of a sub-class. For instance:
class A
class B extends A
class C extends B
scala> Option(new B)
res196: Option[B] = Some(B#661f82ac)
scala> res196.getOrElse(new C)
res197: B = B#661f82ac
scala> res196.getOrElse(new A)
res198: A = B#661f82ac
scala> res196.getOrElse("...")
res199: Object = B#661f82ac
I can still pass an instance of C, because C can be up-cast to B. I can also pass a type higher up the inheritance tree, and getOrElse will return that type, instead. If I pass a type that has nothing to do with the type contained in the Option, then the type with the least upper-bound will be inferred. In the above case, it's Any.
So why is the lower-bound there at all? Why not have:
def getOrElse[B <: A](default: => B): B
This won't work because getOrElse must either return the A that's contained in the Option, or the default B. But if we return the A, and A is not a B, so the type-bound is invalid. Perhaps if getOrElse returned A:
def getOrElse[B <: A](default: => B): A
This would work (if it were really defined that way), but you would be restricted by the type-bounds. So in my above example, you could only pass B or C to getOrElse on an Option[B]. In any case, this is not how it is in the standard library.
The standard library getOrElse allows you to pass anything to it. Say you have Option[A]. If we pass a sub-type of A, then it is up-cast to A. If we pass A, obviously this is okay. And if we pass some other type, then the compiler infers the least upper-bound between the two. In all cases, the type-bound B >: A is met.
Because getOrElse allows you to pass anything to it, many consider it very tricky. For example you could have:
val number = "blah"
// ... lots of code
val result = Option(1).getOrElse(number)
And this will compile. We'll just have an Option[Any] that will probably cause an error somewhere else down the line.
The Scala Language Specification (Section 4.5 on Variance Annotations, p. 44) says
The variance position of a type parameter is the opposite of the variance position
of the enclosing type parameter clause.
The variance position of the lower bound of a type declaration or type parameter
is the opposite of the variance position of the type declaration or parameter.
Using the first point above, it is easy to see (at least formally) that
trait Covariant[+A] {
def problematic[B <: A](x : B)
}
produces the error message
error: covariant type A occurs in contravariant position in type >: Nothing <: A of type B
def problematic[B <: A](x : B)
and using the first and the second point it is easy to see that
trait Contravariant[-A] {
def problematic[B >: A](x : B)
}
produces the error message
error: contravariant type A occurs in covariant position in type >: A <: Any of type B
def problematic[B >: A](x : B)
As I mention, it's easy to see formally (i.e., following the rules for variance annotations) why these errors occur. However, I can not come up with an example illustrating the need for these restrictions. In contrast, it is very easy to come up with examples that illustrate why method parameters should change variance positions, see e.g. Checking Variance Annotations.
So, my question is the following: Assuming, the two pieces of codes above were allowed, what are the examples of problems that arise? This means, I'm looking for examples similar to this one that illustrate what could go wrong in case the two rules cited above were not used. I'm particularly interested in the example involving lower type bounds.
Note that the answer to Scala type bounds & variance leaves this particular question open, whereas the answer given in The "lower bound" will reverse the variance of a type, but why? seems wrong to me.
Edit: I think the first case can be handled as follows (adapting the example cited above). Assume, the following was allowed
trait Queue[+T] {
def head : T
def tail : Queue[T]
def enqueue[U <: T](x : U) : Queue[T]
}
Then we could implement
class QueueImplementation[+T] extends Queue[T] {
/* ... implement Queue here ... */
}
class StrangeIntQueue extends QueueImplementation[Int] {
override def enqueue[U <: Int](x : U) : Queue[Int] = {
println(math.sqrt(x))
super.enqueue(x)
}
}
and use it as
val x : Queue[Any] = new StrangeIntQueue
x.enqueue("abc")
which is clearly troublesome. However, I can not see how to adapt this in order to show that the combination "contravariant type parameter + lower type bound" is also problematic?
Let's suppose we allow for a class to have a type parameter [-T] and a method on that class to have [U >: T]...
for come class hierarchy
Dog <: Mammal <: Animal
class Contra[-X](x: X){
def problem[Y >: X](y: Y): Y = x // X<:Y so this would be valid
}
val cMammal:Contra[Mammal] = new Contra(new Mammal)
val a:Animal = cMammal problem new Animal // Animal >: Mammal, this is fine
val m:Mammal = cMammal problem new Mammal // Mammal >: Mammal, this is fine
val d:Mammal = cMammal problem new Dog // (Dog upcasts to Mammal) >: Mammal, this is fine
val cDog:Contra[Dog] = cMammal // Valid assignment
val a:Animal = cDog problem new Animal // Animal >: Mammal, this is fine
val m:Mammal = cDog problem new Mammal // Mammal >: Mammal, this is fine
val d:Dog = cDog problem new Dog // AAAHHHHHHH!!!!!!
This last line would type check, cDog problem new Dog would actually return a Mammal. This is clearly not a good thing. Thankfully the type system doesn't actually let us do this.
Q.E.D. contravariant type parameter + lower type bound not a good idea to mix.
I hope this example helps.
Use the ++ method from List to see why the restrictions are needed. Due note, this requires that ++ produce a List[B]:
def ++[B](that: GenTraversableOnce[B]): List[B]
with a full signature of
def ++[B >: A, That](that: GenTraversableOnce[B])(implicit bf: CanBuildFrom[List[A], B, That]): That
So why is it important that [B >: A]. Well, what if we want to combine something such that
trait Foo
trait Bar extends Foo
and we have a method that has a signature
def op(that: List[Foo], other: Foo): List[Foo] = that ++ List(other)
I can pass it a list of type Bar but in order to be able to return it as a List[Foo] I must make the condition that Foo >: Bar so that I can actually do the following
def see(that: List[Bar]): List[Foo] = op(that, myFoo)
which essentially is doing a List[Bar] ++ List[Foo] to return a type of List[Foo] as expressed though a List[Foo] type. That is why the flip happens.
Now if I tried to enforce that Foo <: Bar I would immediately run into the issue that List[Bar] ++ List[Foo] could not return a list of type Foo (not to mention having it conflict with the definition above.) It would only ever be able to return a List of the least upper bound.