What is the difference between List[T] forSome {type T} and List[T forSome {type T}]? How do I read them in "English"? How should I grok the forSome keyword? What are some practical uses of forSome? What are some useful practical and more complex than simple T forSome {type T} usages?
Attention: (Update 2016-12-08) The forSome keyword is very likely going away with Scala 2.13 or 2.14, according to Martin Odersky's talk on the ScalaX 2016. Replace it with path dependent types or with anonymous type attributes (A[_]). This is possible in most cases. If you have an edge case where it is not possible, refactor your code or loosen your type restrictions.
How to read "forSome" (in an informal way)
Usually when you use a generic API, the API guarantees you, that it will work with any type you provide (up to some given constraints). So when you use List[T], the List API guarantees you that it will work with any type T you provide.
With forSome (so called existentially quantified type parameters) it is the other way round. The API will provide a type (not you) and it guarantees you, it will work with this type it provided you. The semantics is, that a concrete object will give you something of type T. The same object will also accept the things it provided you. But no other object may work with these Ts and no other object can provide you with something of type T.
The idea of "existentially quantified" is: There exists (at least) one type T (in the implementation) to fulfill the contract of the API. But I won't tell you which type it is.
forSome can be read similar: For some types T the API contract holds true. But it is not necessary true for all types T. So when you provide some type T (instead of the one hidden in the implementation of the API), the compiler cannot guarantee that you got the right T. So it will throw a type error.
Applied to your example
So when you see List[T] forSome {type T} in an API, you can read it like this: The API will provide you with a List of some unknown type T. It will gladly accept this list back and it will work with it. But it won't tell you, what T is. But you know at least, that all elements of the list are of the same type T.
The second one is a little bit more tricky. Again the API will provide you with a List. And it will use some type T and not tell you what T is. But it is free to choose a different type for each element. A real world API would establish some constraints for T, so it can actually work with the elements of the list.
Conclusion
forSome is useful, when you write an API, where each object represents an implementation of the API. Each implementation will provide you with some objects and will accept these objects back. But you can neither mix objects from different implementations nor can you create the objects yourself. Instead you must always use the corresponding API functions to get some objects that will work with that API. forSome enables a very strict kind of encapsulation. You can read forSome in the following way:
The API contract folds true for some types. But you don't know for
which types it holds true. Hence you cannot provide you own type and
you cannot create your own objects. You have to use the ones provided
through the API that uses forSome.
This is quite informal and might even be wrong in some corner cases. But it should help you to grok the concept.
There are a lot of questions here, and most of them have been addressed pretty thoroughly in the answers linked in the comments above, so I'll respond to your more concrete first question.
There's no real meaningful difference between List[T] forSome { type T } and List[T forSome { type T }], but we can see a difference between the following two types:
class Foo[A]
type Outer = List[Foo[T]] forSome { type T }
type Inner = List[Foo[T] forSome { type T }]
We can read the first as "a list of foos of T, for some type T". There's a single T for the entire list. The second, on the other hand, can be read as "a list of foos, where each foo is of T for some T".
To put it another way, if we've got a list outer: Outer, we can say that "there exists some type T such that outer is a list of foos of T", where for a list of type Inner, we can only say that "for each element of the list, there exists some T such that that element is a foo of T". The latter is weaker—it tells us less about the list.
So, for example, if we have the following two lists:
val inner: Inner = List(new Foo[Char], new Foo[Int])
val outer: Outer = List(new Foo[Char], new Foo[Int])
The first will compile just fine—each element of the list is a Foo[T] for some T. The second won't compile, since there's not some T such that each element of the list is a Foo[T].
Related
I would like to use the cats-saga from that repository: https://github.com/VladKopanev/cats-saga
However I am stuck on that piece of code at OrderSagaCoordinator.scala L160:
def apply[F[_]: Sync: Concurrent: Timer: Sleep: Parallel](
paymentServiceClient: PaymentServiceClient[F],
loyaltyPointsServiceClient: LoyaltyPointsServiceClient[F],
orderServiceClient: OrderServiceClient[F],
sagaLogDao: SagaLogDao[F],
maxRequestTimeout: Int
): F[OrderSagaCoordinatorImpl[F]] =
What is F, where does it come from, can someone explain that piece of code ?
Thanks
Edit: I know what a generic type is. However in that case the apply method is called without specifying the concrete type and I do not see any places where it came from.
(for {
paymentService <- PaymentServiceClientStub(randomUtil, clientMaxReqTimeout, flakyClient)
loyaltyPoints <- LoyaltyPointsServiceClientStub(randomUtil, clientMaxReqTimeout, flakyClient)
orderService <- OrderServiceClientStub(randomUtil, clientMaxReqTimeout, flakyClient)
xa = Transactor.fromDriverManager[IO]("org.postgresql.Driver", "jdbc:postgresql:Saga", "postgres", "root")
logDao = new SagaLogDaoImpl(xa)
orderSEC <- OrderSagaCoordinatorImpl(paymentService, loyaltyPoints, orderService, logDao, sagaMaxReqTimeout)
// ...
Think of something concrete, say 'box of chocolates'
case class Box(v: Chocolate)
Now imagine we take away the chocolate, and make the box take any kind of element A, maybe box of coins, box of candy, box of cards, etc
case class Box[A](v: A)
Here we are polymorphic in the element type of the box. Many languages can express this level of polymorphism. But Scala takes this further. In the same way how we took away the chocolate, we can take away the box itself, essentially expressing a very abstract idea of "any kind of context of any type of elements"
trait Ctx[F[_]]
As another analogy consider the following
box of chocolate -> proper type -> case class Box(v: Chocolate)
box of _ -> type constructor of first order -> case class Box[A](v: A)
_ of _ -> type constructor of higher order -> trait Ctx[F[_]]
Now focus on _ of _. Here we have "something of something", which kind of seems like we have nothing. How can we do anything with this? This is where the idea of a type class comes into play. A type class can constrain a highly polymorphic shape such as F[_]
def apply[F[_]: Sync](...)
Here [F[_]: Sync] represents this constraint. It means that method apply accepts any type constructor of first kind for which there exists evidence that it satisfies the constraints of type class Sync. Note that type class Sync
trait Sync[F[_]]
is considered a higher order type constructor, whilst type parameter F[_] represents a first order type constructor. Similarly
F[_] : Sync : Concurrent
specifies that type constructor F must not only satisfy Sync constraints, but also constraints of Concurrent type class, and so on. These techniques are sometimes referred to as scary sounding
higher order type constructor polymorphism
and yet I am confident that most programmers have all the conceptual tools already present to understand it because
if you ever passed a function to a function, then you can work with concept of higher order
if you ever used a List, then you can work with concept of type constructors
if you ever wrote a method that uses the same implementation for both Integers and Doubles, then you can work with concept of polymorphism
Providing evidence that a type constructor satisfies constraints of a type class are given using Scala's implicit mechanisms. IMO Scala 3 has significantly simplified the concept so consider https://dotty.epfl.ch/docs/reference/contextual/type-classes.html
I wonder if anyone could explain the inferencing rule in this particular case below, and most importantly it's rational/implication ?
case class E[A, B](a: A) // class E
E(2) // E[Int,Nothing] = E(2)
Note that I could have wrote E[Int](2). What matter to me is why is the second parameter type inferred to be Nothing (i.e. Bottom type) instead of let say Any for instance ? Why is that and What's the rational/Implication ?
Just to give some context, this is related to the definition of Either and how it works for Left and Right. Both are defined according to the pattern
final case class X[+A, +B](value: A) extends Either[A, B]
Where you instantiate it let say as Right[Int](2) and the type inferred is Right[Nothing, Int] and by extension Either[Nothing, Int]
EDIT1
There is consistency here, but i still can figure out the rational. Below is the same definition with a contra-variant paramete:
case class E[A, -B](a: A)// class E
E(2) // E[Int, Any] = E(2)
Hence we do have the same thing the other way around when it is contra-variant, and that make the all behavior or inference rule, coherent. However the rational for this i am not sure ....
Why not the opposite rule i.e. infer Any when Co-Variant/Invariant and Nothing when Contra-Variant ?
EDIT2
In the light of #slouc Answer, which make good sense, i'm left with still understanding what and why the compiler is doing what it is doing. The example below illustrate my confusion
val myleft = Left("Error") // Left[String,Nothing] = Left(Error)
myleft map { (e:Int) => e * 4} // Either[String,Int] = Left(Error)
First the compiler fix the type to something that "for sure work" to reuse the conclusion of #slouc (albeit make more sense in the context of a Function) Left[String,Nothing]
Next the compile infer myleft to be of type Either[String,Int]
given map definition def map[B](f: A => B): Either[E, B], (e:Int) => e * 4 can only be supplied if myleft is actually Left[String,Int] or Either[String,Int]
So in other words, my question is, what is the point of fixing the type to Nothing if it is to change it later.
Indeed the following does not compile
val aleft: Left[String, Nothing] = Left[String, Int]("Error")
type mismatch;
found : scala.util.Left[String,Int]
required: Left[String,Nothing]
val aleft: Left[String, Nothing] = Left[String, Int]("Error")
So why would I infer to a type, that normally would block me to do anything else over variable of that type (but for sure works in term of inference), to ultimately change that type, so i can do something with a variable of that inferred type.
EDIT3
Edit2 is a bit misunderstanding and everything is clarified in #slouc answer and comments.
Covariance:
Given type F[+A] and relation A <: B, then the following holds: F[A] <: F[B]
Contravariance:
Given type F[-A] and relation A <: B, then the following holds: F[A] >: F[B]
If the compiler cannot infer the exact type, it will resolve the lowest possible type in case of covariance and highest possible type in case of contravariance.
Why?
This is a very important rule when it comes to variance in subtyping. It can be shown on the example of the following data type from Scala:
trait Function1[Input-, Output+]
Generally speaking, when a type is placed in the function/method parameters, it means it's in the so-called "contravariant position". If it's used in function/method return values, it's in the so-called "covariant position". If it's in both, then it's invariant.
Now, given the rules from the beginning of this post, we conclude that, given:
trait Food
trait Fruit extends Food
trait Apple extends Fruit
def foo(someFunction: Fruit => Fruit) = ???
we can supply
val f: Food => Apple = ???
foo(f)
Function f is a valid substitute for someFunction because:
Food is a supertype of Fruit (contravariance of input)
Apple is a subtype of Fruit (covariance of output)
We can explain this in natural language like this:
"Method foo needs a function that can take a Fruit and produce a
Fruit. This means foo will have some Fruit and will need a
function it can feed it to, and expect some Fruit back. If it gets a
function Food => Apple, everything is fine - it can still feed it
Fruit (because the function takes any food), and it can receive
Fruit (apples are fruit, so the contract is respected).
Coming back to your initial dilemma, hopefully this explains why, without any extra information, compiler will resort to lowest possible type for covariant types and highest possible type for contravariant ones. If we want to supply a function to foo, there's one that we know surely works: Any => Nothing.
Variance in general.
Variance in Scala documentation.
Article about variance in Scala (full disclosure: I wrote it).
EDIT:
I think I know what's confusing you.
When you instantiate a Left[String, Nothing], you're allowed to later map it with a function Int => Whatever, or String => Whatever, or Any => Whatever. This is precisly because of the contravariance of function input explained earlier. That's why your map works.
"what is the point of fixing the type to Nothing if it is to change it
later?"
I think it's a bit hard to wrap your head around compiler fixing the unknown type to Nothing in case of contravariance. When it fixes the unknown type to Any in case of covariance, it feels more natural (it can be "Anything"). Because of the duality of covariance and contravariance explained earlier, same reasoning applies for contravariant Nothing and covariant Any.
This is a quote from
Unification of Compile-Time and Runtime Metaprogramming in Scala
by Eugene Burmako
https://infoscience.epfl.ch/record/226166 (p. 95-96)
During type inference, the typechecker collects constraints on missing
type arguments from bounds of type parameters, from types of term
arguments, and even from results of implicit search (type inference
works together with implicit search because Scala supports an analogue
of functional dependencies). One can view these constraints as a
system of inequalities where unknown type arguments are represented as
type variables and order is imposed by the subtyping relation.
After collecting constraints, the typechecker starts a step-by-step
process that, on each step, tries to apply a certain transformation to
inequalities, creating an equivalent, yet supposedly simpler system of
inequalities. The goal of type inference is to transform the original
inequalities to equalities that represent a unique solution of the
original system.
Most of the time, type inference succeeds. In that
case, missing type arguments are inferred to the types represented by
the solution.
However, sometimes type inference fails. For example,
when a type parameter T is phantom, i.e. unused in the term parameters
of the method, its only entry in the system of inequalities will be
L <: T <: U, where L and U are its lower and upper bound respectively.
If L != U, this inequality does not have a unique solution, and that
means a failure of type inference.
When type inference fails, i.e.
when it is unable to take any more transformation steps and its
working state still contains some inequalities, the typechecker breaks
the stalemate. It takes all yet uninferred type arguments, i.e. those
whose variables are still represented by inequalities, and forcibly
minimizes them, i.e. equates them to their lower bounds. This produces
a result where some type arguments are inferred precisely, and some
are replaced with seemingly arbitrary types. For instance,
unconstrained type parameters are inferred to Nothing, which is a
common source of confusion for Scala beginners.
You can learn more about type inference in Scala:
Hubert Plociniczak Decrypting Local Type Inference https://infoscience.epfl.ch/record/214757
Guillaume Martres Scala 3, Type Inference and You! https://www.youtube.com/watch?v=lMvOykNQ4zs
Guillaume Martres Dotty and types: the story so far https://www.youtube.com/watch?v=YIQjfCKDR5A
Slides http://guillaume.martres.me/talks/
Aleksander Boruch-Gruszecki GADTs in Dotty https://www.youtube.com/watch?v=VV9lPg3fNl8
I've been digging into FP and everything that surrounds it, and I found the concept of kind projector written somewhere, without details nor explanations.
The only thing I found was this github project, and I'm starting to think if it was referring to this particular project, or to some generic concept in FP?
So, what is a kind projector? Why is it useful? (if possible, can you provide examples, resources, etc?)
This is indeed just a slightly awkward name for the specific plugin for the Scala compiler you linked to. I don't think it has any significance to itself, but it kind of fits its purpose.
What the plugin does is to provide an alternative syntax to Scala's usual workaround for type lambdas, which uses a language feature called type projections.
Say you wanted to implement Functor for Either. Now, Functor requires kind * -> *, whereas Either has kind * -> * -> *. So we need to first fix the first argument, and can then provide the implementation for the partially applied type constructor. The only way you can do this in "regular" Scala is this:
implicit def eitherIsFunctor[A]: Functor[{type λ[X] = Either[A, X]}#λ] = { ... }
where {type λ[X] = Either[A, X]} is an anonymous structural type, which is only immediately used to "project out" λ, the type we actually want. In Haskell, you could just say
instance Functor (Either a) where ...
where Either is partially applied (and a is quantified over automatically).
The plugin allows one to replace the projection with something that looks more like a usual partial application in Scala, namely Either[A, ?], instead of the hardly understandable {type λ[X] = Either[A, X]}#λ (and also provides general type lambdas, I think, always by converting them down to anonymous types and projections).
Scala 3 provides native type lambdas which are no longer based on type projection
A type lambda such as [X] =>> F[X] defines a function from types to
types.
For example,
trait Functor[F[_]]
new Functor[Either[String, Int]] {} // error
new Functor[({ type λ[X] = Either[String, X] })#λ] {} // Scala 2 type lambda based on type projection
new Functor[λ[X => Either[String, X]]] {} // kind projector type lambda
new Functor[Either[String, *]] {} // kind projector type lambda
new Functor[[X] =>> Either[String, X]] {} // Scala 3 type lambda
Also, there exists a proposal SIP: Underscore Syntax for Type Lambdas #5379 such that
Functor[Either[String, _]] // equivalent to Functor[[X] =>> Either[String, X]]
Disclaimer: I'm not sure I really understand type context bounds... or anything.
Is it possible to define a type in scala that is an instance of a TypeClass where the type can be Any, so long as that type obeys a type context bound.
For example I have some typeclass Printable[T]
Printable is a typeclass for things that can be printed, and it's useful as a type context bound because I might want to have typeclass that can take any type as long as it's printable, like this:
class SomeContainer[T: Printable](value: T)
From what I understand the collections library uses type context bounds like this for things that can be ordered, summed etc.
But what I want is type PrintableAnyCollection = SomeCollection[_: Printable]
That is a collection of values that can be of different types, as long as all those types all obey to type context bound that there exists a Printable[T] for w/e that type.
TLDR:
Collection[Any] almost does what I want because it can hold different types
Collection[T: Printable] Almost does what I want because it enforces that the things in the collection are printable but has the consequence that the collection stores only the one type.
Collection[_: Printable] and/or Collection[Any: Printable] look like they sort of describe what I want but aren't valid syntax.
You're presumably looking for something like this:
// Doesn't work
Collection[T forSome {type T: Printable}]
This doesn't work though and the reason is because context bounds are just sugar for implicit parameters, which are resolved at compile time. In particular, the above type would imply that the implicit resolutions the compiler did would depend on the runtime value of the above type (each different T would need another implicit resolution). In effect you would create a method whose arity in its implicit parameters list could change at runtime.
Suffice it to say that doesn't play nice with Scala.
An alternative is to proactively bundle the implicit parameter in some sort of wrapper type and parameterize your collection on the wrapper as an existential. Thus implicit resolution occurs before the collection is created. Miles Sabin covers it well here.
To briefly summarize that approach, it looks something like the following:
case class PrintableWrapper[T](unwrap: T)(implicit ev: Printable[T])
Collection[PrintableWrapper[T] forSome {type T}]
Actually using the entries of the collection is cumbersome (a spurious pattern match is required to appease Scala's typechecker). The desugared context bound is also required to have an explicit instance of Printable to actually print the entry. See Miles Sabin's answer for more details.
This happens to be one spot where implicits are a slightly cumbersome way of encoding typeclasses.
I found the following code snippet:
List[T] forSome { type T }
The forSome looks like a method, but my friend told me it's a keyword.
I googled it, but found few documents about forSome. What does it mean, and where can I get some documents about it?
The forSome keyword is used to define existential types in Scala. There's this Scala glossary page explaining what they are. I couldn't find a place in the Scala docs explaining them in detail, so here is a blog article I found on Google explaining how they are useful.
Update: you can find a precise definition of existential types in the Scala specification but it is quite dense.
To summarize some of the posts I linked to, existential types are useful when you want to operate on something but don't care about the details of the type in it. For example, you want to operate on arrays but don't care what kind of array:
def printFirst(x : Array[T] forSome {type T}) = println(x(0))
which you could also do with a type variable on the method:
def printFirst[T](x : Array[T]) = println(x(0))
but you may not want to add the type variable in some cases. You can also add a bound to the type variable:
def addToFirst(x : Array[T] forSome {type T <: Integer}) = x(0) + 1
Also see this blog post which is where I got this example from.
I don't know Scala, but your question picked up my interest and started Googling.
I found that in Scala's changelog:
"It is now possible to define existential types using the new keyword
forSome. An existential type has the form T forSome {Q} where Q is a
sequence of value and/or type declarations. "