I'm going throught collection api and the returned type of operations which can modify the type of container and the type of contained element have all the header in its non-usecase form similar to the following:def map [B, That] (f: (A) ⇒ B)(implicit bf: CanBuildFrom[List[A], B, That]):That
It's clear that there is sure 'fallback' to other collection type with That (as in case of BitSet fallbacking to Set if the contained item is changed to other than Int). But how can one be sure of the type of the item? I would expect the return type to That[B], but it is not. Why? Educate me please :-)
You already gave the answer yourself: If you want to be able to return a BitSet in appropriate cases, you need to get rid of that type parameter in the return type. There is no BitSet[Int], obviously.
All the information you need is already handled by the appropriate CanBuildFrom implicits which have a fallback definition of the type CanBuildFrom[Coll, A, Traversable[A]] (or whatever may be over Traversable in the type hierarchy).
Of course, the CanBuildFrom must be sensible but I think it’s actually possible to abuse it and return a truly strange That. Nonetheless, the defaults are pretty sane.
Related
In the book Functional Programming in Scala MEAP v10, the author mentions
Polymorphic functions are often so constrained by their type that they only have one implementation!
and gives the example
def partial1[A,B,C](a: A, f: (A,B) => C): B => C = (b: B) => f(a, b)
What does he mean by this statement? Are polymorphic functions restrictive?
Here's a simpler example:
def mysteryMethod[A, B](somePair: (A, B)): B = ???
What does this method do? It turns out, that there is only one thing this method can do! You don't need the name of the method, you don't need the implementation of the method, you don't need any documentation. The type tells you everything it could possibly do, and it turns out that "everything" in this case is exactly one thing.
So, what does it do? It takes a pair (A, B) and returns some value of type B. What value does it return? Can it construct a value of type B? No, it can't, because it doesn't know what B is! Can it return a random value of type B? No, because randomness is a side-effect and thus would have to appear in the type signature. Can it go out in the universe and fetch some B? No, because that would be a side-effect and would have to appear in the type signature!
In fact, the only thing it can do is return the value of type B that was passed into it, the second element of the pair. So, this mysteryMethod is really the second method, and its only sensible implementation is:
def second[A, B](somePair: (A, B)): B = somePair._2
Note that in reality, since Scala is neither pure nor total, there are in fact a couple of other things the method could do: throw an exception (i.e. return abnormally), go into an infinite loop (i.e. not return at all), use reflection to figure out the actual type of B and reflectively invoke the constructor to fabricate a new value, etc.
However, assuming purity (the return value may only depend on the arguments), totality (the method must return a value normally) and parametricity (it really doesn't know anything about A and B), then there is in fact an awful lot you can tell about a method by only looking at its type.
Here's another example:
def mysteryMethod(someBoolean: Boolean): Boolean = ???
What could this do? It could always return false and ignore its argument. But then it would be overly constrained: if it always ignores its argument, then it doesn't care that it is a Boolean and its type would rather be
def alwaysFalse[A](something: A): Boolean = false // same for true, obviously
It could always just return its argument, but again, then it wouldn't actually care about booleans, and its type would rather be
def identity[A](something: A): A = something
So, really, the only thing it can do is return a different boolean than the one that was passed in, and since there are only two booleans, we know that our mysteryMethod is, in fact, not:
def not(someBoolean: Boolean): Boolean = if (someBoolean) false else true
So, here, we have an example, where the types don't give us the implementation, but at least, they give as a (small) set of 4 possible implementations, only one of which makes sense.
(By the way: it turns out that there is only one possible implementation of a method which takes an A and returns an A, and it is the identity method shown above.)
So, to recap:
purity means that you can only use the building blocks that were handed to you (the arguments)
a strong, strict, static type system means that you can only use those building blocks in such a way that their types line up
totality means that you can't do stupid things (like infinite loops or throwing exceptions)
parametricity means that you cannot make any assumptions at all about your type variables
Think about your arguments as parts of a machine and your types as connectors on those machine parts. There will only be a limited number of ways that you can connect those machine parts together in a way that you only plug together compatible connectors and you don't have any leftover parts. Often enough, there will be only one way, or if there are multiple ways, then often one will be obviously the right one.
What this means is that, once you have designed the types of your objects and methods, you won't even have to think about how to implement those methods, because the types will already dictate the only possible way to implement them! Considering how many questions on StackOverflow are basically "how do I implement this?", can you imagine how freeing it must be not having to think about that at all, because the types already dictate the one (or one of a few) possible implementation?
Now, look at the signature of the method in your question and try playing around with different ways to combine a and f in such a way that the types line up and you use both a and f and you will indeed see that there is only one way to do that. (As Chris and Paul have shown.)
def partial1[A,B,C](a: A, f: (A,B) => C): B => C = (b: B) => f(a, b)
Here, partial1 takes as parameters value of type A, and a function that takes a parameter of type A and a parameter of type B, returning a value of type C.
partial1 must return a function taking a value of type B and returning a C. Given A, B, and C are arbitary, we cannot apply any functions to their values. So the only possibility is to apply the function f to the value a passed to partial, and the value of type B that is a parameter to the function we return.
So you end up with the single possibility that's in the definition f(a,b)
To take a simpler example, consider the type Option[A] => Boolean. There's only a couple ways to implement this:
def foo1(x: Option[A]): Boolean = x match { case Some(_) => true
case None => false }
def foo2(x: Option[A]): Boolean = !foo1(x)
def foo3(x: Option[A]): Boolean = true
def foo4(x: Option[A]): Boolean = false
The first two choices are pretty much the same, and the last two are trivial, so essentially there's only one useful thing this function could do, which is tell you whether the Option is Some or None.
The space of possible implementation is "restricted" by the abstractness of the function type. Since A is unconstrained, the option's value could be anything, so the function can't depend on that value in any way because you know nothing about what it. The only "understanding" the function may have about its parameter is the structure of Option[_].
Now, back to your example. You have no idea what C is, so there's no way you can construct one yourself. Therefore the function you create is going to have to call f to get a C. And in order to call f, you need to provide an arguments of types A and B. Again, since there's no way to create an A or a B yourself, the only thing you can do is use the arguments that are given to you. So there's no other possible function you could write.
What is the difference between List[T] forSome {type T} and List[T forSome {type T}]? How do I read them in "English"? How should I grok the forSome keyword? What are some practical uses of forSome? What are some useful practical and more complex than simple T forSome {type T} usages?
Attention: (Update 2016-12-08) The forSome keyword is very likely going away with Scala 2.13 or 2.14, according to Martin Odersky's talk on the ScalaX 2016. Replace it with path dependent types or with anonymous type attributes (A[_]). This is possible in most cases. If you have an edge case where it is not possible, refactor your code or loosen your type restrictions.
How to read "forSome" (in an informal way)
Usually when you use a generic API, the API guarantees you, that it will work with any type you provide (up to some given constraints). So when you use List[T], the List API guarantees you that it will work with any type T you provide.
With forSome (so called existentially quantified type parameters) it is the other way round. The API will provide a type (not you) and it guarantees you, it will work with this type it provided you. The semantics is, that a concrete object will give you something of type T. The same object will also accept the things it provided you. But no other object may work with these Ts and no other object can provide you with something of type T.
The idea of "existentially quantified" is: There exists (at least) one type T (in the implementation) to fulfill the contract of the API. But I won't tell you which type it is.
forSome can be read similar: For some types T the API contract holds true. But it is not necessary true for all types T. So when you provide some type T (instead of the one hidden in the implementation of the API), the compiler cannot guarantee that you got the right T. So it will throw a type error.
Applied to your example
So when you see List[T] forSome {type T} in an API, you can read it like this: The API will provide you with a List of some unknown type T. It will gladly accept this list back and it will work with it. But it won't tell you, what T is. But you know at least, that all elements of the list are of the same type T.
The second one is a little bit more tricky. Again the API will provide you with a List. And it will use some type T and not tell you what T is. But it is free to choose a different type for each element. A real world API would establish some constraints for T, so it can actually work with the elements of the list.
Conclusion
forSome is useful, when you write an API, where each object represents an implementation of the API. Each implementation will provide you with some objects and will accept these objects back. But you can neither mix objects from different implementations nor can you create the objects yourself. Instead you must always use the corresponding API functions to get some objects that will work with that API. forSome enables a very strict kind of encapsulation. You can read forSome in the following way:
The API contract folds true for some types. But you don't know for
which types it holds true. Hence you cannot provide you own type and
you cannot create your own objects. You have to use the ones provided
through the API that uses forSome.
This is quite informal and might even be wrong in some corner cases. But it should help you to grok the concept.
There are a lot of questions here, and most of them have been addressed pretty thoroughly in the answers linked in the comments above, so I'll respond to your more concrete first question.
There's no real meaningful difference between List[T] forSome { type T } and List[T forSome { type T }], but we can see a difference between the following two types:
class Foo[A]
type Outer = List[Foo[T]] forSome { type T }
type Inner = List[Foo[T] forSome { type T }]
We can read the first as "a list of foos of T, for some type T". There's a single T for the entire list. The second, on the other hand, can be read as "a list of foos, where each foo is of T for some T".
To put it another way, if we've got a list outer: Outer, we can say that "there exists some type T such that outer is a list of foos of T", where for a list of type Inner, we can only say that "for each element of the list, there exists some T such that that element is a foo of T". The latter is weaker—it tells us less about the list.
So, for example, if we have the following two lists:
val inner: Inner = List(new Foo[Char], new Foo[Int])
val outer: Outer = List(new Foo[Char], new Foo[Int])
The first will compile just fine—each element of the list is a Foo[T] for some T. The second won't compile, since there's not some T such that each element of the list is a Foo[T].
everyone . Please forgive me asking a stupid question on Scala.
Though I have been programming in Scala for about 2 years, I still find it hard to understand implicit usage. Let's take an example for discussion:
Array(1,2,3,4).map(x => x)
If you look up the scaladoc, you cant' find the method map on Array class. The reason that map can apply on Array(1,2,3,4) is that there is an implicit function implicit def intArrayOps (xs: Array[Int]): ArrayOps[Int] defined in scala.Predef.
However, there are two parameter lists, where the second one is written as implicit bf: CanBuildFrom[Array[T], B, That]). Now I wonder where the compiler finds a proper argument for type CanBuildFrom when applying map on Array(1,2,3,4).
The implicit resolution includes searching the companion object for the type of the implicit parameter as well as the companion objects for the type parameters of the implicit parameter. In the example above the signature of map is the following
def map[B, That](f: (Int) => B)(implicit bf: CanBuildFrom[Array[Int], B, That]): That
Since we have no type requirements for That we can ignore it for now. After we look in the local and container scopes and find no matching implicits, the next place to look for an implicit would be the companion object for CanBuildFrom. However it has no companion object. So we continue on and look in Array for an implicit. We find one in the form of
implicit def canBuildFrom[T](implicit m: ClassManifest[T]): CanBuildFrom[Array[_], T, Array[T]]
Since we have no type requirements and a matching implicit, "That" is forced to be of type Array[Int] and completes our typing.
This question is partially answered with other question on StackOverflow. Let me give a try to summarize them:
The first part you need to know is where the Scala compiler looks for implicits. You can find some more detail about CanBuildFrom here.
If you have understood what is mentioned in the answers about implicits ‣you should take a look to the construction of Scala Collections. Their inheritance-hierarchy is explained here and for List here. All of them are build up with Builders. This is explained in detail in a question about breakOut.
To round up your knowledge, you should know how to pimp the Collections. Also, this is explained on StackOverflow in this question.
Please note, the best answers on StackOverflow are summarized in the Scala-Tag-Wiki.
According to the documentation, PartiallyOrdered[A] is covariant in A, while Ordered[A] is invariant (but used to be covariant) in A.
Why was Ordered[A] ever covariant in A? Isn't this an obvious violation of the substitution principle?
Why can't Ordered[A] be contravariant in A? This would allow an Ordered[Traversible[Char]] to be typed as an Ordered[StringBuilder], for example. I don't see how this could be problematic.
I'm having trouble understanding the signature of tryCompareTo in PartiallyOrdered. It looks like the argument can be an instance of any supertype of A. Couldn't you pass in any object by calling tryCompareTo[Any](anything)? If so, how is the method signature any better than tryCompareTo(that: Any)?
Logically, ordered sets are a subclass of partially ordered sets, but the Scala classes don't seem to reflect this relationship. Is this because Ordered[A] cannot be covariant in A as PartiallyOrdered[A] can?
Looking at the previous version (2.8.1) , I didn't find anything relevant about Ordered that was covariant ?
EDIT : Looked further and find some explanation in the documentation for Ordered to not be covariant anymore.
For the case of tryCompare in PartiallyOrdered, the signature set more restriction on passed parameter :
def tryCompareTo [B >: A] (that: B)(implicit arg0: (B) ⇒ PartiallyOrdered[B]): Option[Int]
Here the implicit arg0 implies that to be seen as a PartiallyOrdered[B] instance and thus that it has access to all method defined in this trait.
Inspired by Real-world examples of co- and contravariance in Scala I thought a better question would be:
When designing a library, are there a specific set of questions you should ask yourself when determining whether a type parameter should be covariant or contravariant? Or should you make everything invariant and then change as needed?
Well, simple, does it make sense? Think of Liskov substitution.
Co-variance
If A <: B, does it make sense to pass a C[A] where a C[B] is expected? If so, make it C[+T]. The classic example is the immutable List, where a List[A] can be passed to anything expecting a List[B], assuming A is a subtype of B.
Two counter examples:
Mutable sequences are invariant, because it is possible to have type safety violations otherwise (in fact, Java's co-variant Array is vulnerable to just such things, which is why it is invariant in Scala).
Immutable Set is invariant, even though its methods are very similar to those of an immutable Seq. The difference lies with contains, which is typed on sets and untyped (ie, accept Any) on sequences. So, even though it would otherwise be possible to make it co-variant, the desire for an increased type safety on a particular method led to a choice of invariance over co-variance.
Contra-variance
If A <: B, does it make sense to pass a C[B] where a C[A] is expected? If so, make it C[-T]. The classic would-be example is Ordering. While some unrelated technical problems prevent Ordering from being contra-variant, it is intuitive that anything that can order a super-class of A can also order A. It follows that Ordering[B], which orders all elements of type B, a supertype of A, can be passed to something expecting an Ordering[A].
While Scala's Ordering is not contra-variant, Scalaz's Order is contra-variant as expected. Another example from Scalaz is its Equal trait.
Mixed Variance?
The most visible example of mixed variance in Scala is Function1 (and 2, 3, etc). It is contra-variant in the parameter it receives, and co-variant in what it returns. Note, though, that Function1 is what is used for a lot of closures, and closures are used in a lot of places, and these places are usually where Java uses (or would use) Single Abstract Method classes.
So, if you have a situation where a SAM class applies, that's likely a place for mixed contra-variance and co-variance.