At a high-level, I'm curious to know how implicits defined inside a class body are resolved? I assumed they would be considered part of the current scope (as per the awesome answer here), but I am wrong.
For a specific example, why doesn't the following work:
trait Convert[T, U] {
final def value(record: T)(implicit f: T => U): U = f(record)
}
case class ConvertMap(key: String)
extends Convert[Map[String, String], Boolean] {
implicit def a2b(m: Map[String, String]): Boolean = m.contains(key)
}
I can instantiate the class ConvertMap, but when I call the value method I get an error stating that the view from Map[String, String] => Boolean can't be found.
scala> val c = ConvertMap("key")
c: ConvertMap = ConvertMap(key)
scala> c.value(Map("key" -> "value"))
<console>:13: error: No implicit view available from Map[String,String] => Boolean.
If you re-read the answer you provided for implicit resolution scope, you see that there are a couple of things which happen here causing the implicit you've defined not to be found.
First, if the implicit isn't found in the local scope of the call-site, we defer to "category 2" lookup for implicits which involves the types underlying the implicit. We're searching for an implicit conversion of type Map[String, String] => Boolean. According to the implicit scoping rules, the following is applicable to our use case:
If T is a parameterized type S[T1, ..., Tn], the union of the parts of S and T1, ..., Tn.
Our S[T1.. Tn] is a Map[String, String] (and it's base classes), so we have both Map and String as a candidate type for implicit lookup, but neither of them hold the conversion. Further, the return type is also considered, meaning Boolean is also in scope, but again, it doesn't hold the implicit and hence why the compiler complains.
The simplest thing that can be done to help the compiler find it is to place the implicit inside the companion object of ConvertMap, but that means we can no longer accept a value key in the constructor, which makes your example a bit contrived.
It needs to be resolvable at the call site (where c.value is called).
At that time, the only thing you have in scope c. I am not sure why you would think it makes sense for something defined inside a class to be considered in scope at that point.
BTW, your example doesn't really seem like a good use for implicits to begin with. Why not just make f a member method in the trait?
Related
There is a difference in the way the scala 2.13.3 compiler determines which overloaded function to call compared to which overloaded implicit to pick.
object Thing {
trait A;
trait B extends A;
trait C extends A;
def f(a: A): String = "A"
def f(b: B): String = "B"
def f(c: C): String = "C"
implicit val a: A = new A {};
implicit val b: B = new B {};
implicit val c: C = new C {};
}
import Thing._
scala> f(new B{})
val res1: String = B
scala> implicitly[B]
val res2: Thing.B = Thing$$anon$2#2f64f99f
scala> f(new A{})
val res3: String = A
scala> implicitly[A]
^
error: ambiguous implicit values:
both value b in object Thing of type Thing.B
and value c in object Thing of type Thing.C
match expected type Thing.A
As we can see, the overload resolution worked for the function call but not for the implicit pick. Why isn't the implicit offered by val a be chosen as occurs with function calls? If the callers ask for an instance of A why the compilers considers instances of B and C when an instance of A is in scope. There would be no ambiguity if the resolution logic were the same as for function calls.
Edit 2:
The Edit 1 was removed because the assertion I wrote there was wrong.
In response to the comments I added another test to see what happens when the implicit val c: C is removed. In that case the compiler don't complains and picks implicit val b: B despite the caller asked for an instance of A.
object Thing {
trait A { def name = 'A' };
trait B extends A { def name = 'B' };
trait C extends A { def name = 'C' };
def f(a: A): String = "A"
def f(b: B): String = "B"
implicit val a: A = new A {};
implicit val b: B = new B {};
}
import Thing._
scala> f(new A{})
val res0: String = A
scala> implicitly[A].name
val res3: Char = B
So, the overloading resolution of implicit differs from function calls more than I expected.
Anyway, I still don't find a reason why the designers of scala decided to apply a different resolution logic for function and implicit overloading. (Edit: Later noticed why).
Let's see what happens in a real world example.
Suppose we are doing a Json parser that converts a Json string directly to scala Abstract data types, and we want it to support many standard collections.
The snippet in charge of parsing the iterable collections would be something like this:
trait Parser[+A] {
def parse(input: Input): ParseResult;
///// many combinators here
}
implicit def summonParser[T](implicit parserT: Parser[T]) = parserT;
/** #tparam IC iterator type constructor
* #tparam E element's type */
implicit def iterableParser[IC[E] <: Iterable[E], E](
implicit
parserE: Parser[E],
factory: IterableFactory[IC]
): Parser[IC[E]] = '[' ~> skipSpaces ~> (parserE <~ skipSpaces).repSepGen(coma <~ skipSpaces, factory.newBuilder[E]) <~ skipSpaces <~ ']';
Which requires a Parser[E] for the elements and a IterableFactory[IC] to construct the collection specified by the type parameters.
So, we have to put in implicit scope an instance of IterableFactory for every collection type we want to support.
implicit val iterableFactory: IterableFactory[Iterable] = Iterable
implicit val setFactory: IterableFactory[Set] = Set
implicit val listFactory: IterableFactory[List] = List
With the current implicit resolution logic implemented by the scala compiler, this snippet works fine for Set and List, but not for Iterable.
scala> def parserInt: Parser[Int] = ???
def parserInt: read.Parser[Int]
scala> Parser[List[Int]]
val res0: read.Parser[List[Int]] = read.Parser$$anonfun$pursue$3#3958db82
scala> Parser[Vector[Int]]
val res1: read.Parser[Vector[Int]] = read.Parser$$anonfun$pursue$3#648f48d3
scala> Parser[Iterable[Int]]
^
error: could not find implicit value for parameter parserT: read.Parser[Iterable[Int]]
And the reason is:
scala> implicitly[IterableFactory[Iterable]]
^
error: ambiguous implicit values:
both value listFactory in object IterableParser of type scala.collection.IterableFactory[List]
and value vectorFactory in object IterableParser of type scala.collection.IterableFactory[Vector]
match expected type scala.collection.IterableFactory[Iterable]
On the contrary, if the overloading resolution logic of implicits was like the one for function calls, this would work fine.
Edit 3: After many many coffees I noticed that, contrary to what I said above, there is no difference between the way the compiler decides which overloaded functions to call and which overloaded implicit to pick.
In the case of function call: from all the functions overloads such that the type of the argument is asignable to the type of the parameter, the compiler chooses the one such that the function's parameter type is assignable to all the others. If no function satisfies that, a compilation error is thrown.
In the case of implicit pick up: from all the implicit in scope such that the type of the implicit is asignable to the asked type, the compiler chooses the one such that the declared type is asignable to all the others. If no implicit satisfies that, an compilation error is thrown.
My mistake was that I didn't notice the inversion of the assignability.
Anyway, the resolution logic I proposed above (give me what I asked for) is not entirely wrong. It's solves the particular case I mentioned. But for most uses cases the logic implemented by the scala compiler (and, I suppose, all the other languages that support type classes) is better.
As explained in the Edit 3 section of the question, there are similitudes between the way the compiler decides which overloaded functions to call and which overloaded implicit to pick. In both cases the compiler does two steps:
Filters out all the alternatives that are not asignable.
From the remaining alternatives choses the most specific or complains if there is more than one.
In the case of the function call, the most specific alternative is the function with the most specific parameter type; and in the case of implicit pick is the instance with the most specific declared type.
But, if the logic in both cases were exactly the same, then why did the example of the question give different results? Because there is a difference: the assignability requirement that determines which alternatives pass the first step are oposite.
In the case of the function call, after the first step remain the functions whose parameter type is more generic than the argument type; and in the case of implicit pick, remain the instances whose declared type is more specific than the asked type.
The above words are enough to answers the question itself but don't give a solution to the problem that motivated it, which is: How to force the compiler to pick the implicit instance whose declared type is exactly the same than the summoned type? And the answer is: wrapping the implicit instances inside a non variant wrapper.
I am experimenting with a set of custom container functions and took inspiration from Scala's collections library with regard to the CanBuildFrom[-From, -Elem, -To] implicit parameter.
In Scala's collections, CanBuildFrom enables parametrization of the return type of functions like map. Internally the CanBuildFrom parameter is used like a factory: map applies it's first order function on it's input elements, feeds the result of each application into the CanBuildFrom parameter, and finally calls CanBuildFrom's result method which builds the final result of the map call.
In my understanding the -From type parameter of CanBuildFrom[-From, -Elem, -To] is only used in apply(from: From): Builder[Elem, To] which creates a Builder preinitialized with some elements. In my custom container functions I don't have that use case, so I created my own CanBuildFrom variant Factory[-Elem, +Target].
Now I can have a trait Mappable
trait Factory[-Elem, +Target] {
def apply(elems: Traversable[Elem]): Target
def apply(elem: Elem): Target = apply(Traversable(elem))
}
trait Mappable[A, Repr] {
def traversable: Traversable[A]
def map[B, That](f: A => B)(implicit factory: Factory[B, That]): That =
factory(traversable.map(f))
}
and an implementation Container
object Container {
implicit def elementFactory[A] = new Factory[A, Container[A]] {
def apply(elems: Traversable[A]) = new Container(elems.toSeq)
}
}
class Container[A](val elements: Seq[A]) extends Mappable[A, Container[A]] {
def traversable = elements
}
Unfortunately though, a call to map
object TestApp extends App {
val c = new Container(Seq(1, 2, 3, 4, 5))
val mapped = c.map { x => 2 * x }
}
yields an error message not enough arguments for method map: (implicit factory: tests.Factory[Int,That])That. Unspecified value parameter factory . The error goes away when I add an explicit import import Container._ or when I explicitly specify the expected return type val mapped: Container[Int] = c.map { x => 2 * x }
All of these "workarounds" become unnecessary when I add an unused third type parameter to Factory
trait Factory[-Source, -Elem, +Target] {
def apply(elems: Traversable[Elem]): Target
def apply(elem: Elem): Target = apply(Traversable(elem))
}
trait Mappable[A, Repr] {
def traversable: Traversable[A]
def map[B, That](f: A => B)(implicit factory: Factory[Repr, B, That]): That =
factory(traversable.map(f))
}
and change the implicit definition in Container
object Container {
implicit def elementFactory[A, B] = new Factory[Container[A], B, Container[B]] {
def apply(elems: Traversable[A]) = new Container(elems.toSeq)
}
}
So finally my question: Why is the seemingly unused -Source type parameter necessary to resolve the implicit?
And as an additional meta question: How do you solve these kinds of problems if you don't have a working implementation (the collection library) as a template?
Clarification
It might be helpful to explain why I thought the implicit resolution should work without the additional -Source type parameter.
According to this doc entry on implicit resolution, implicits are looked up in companion objects of types. The author does not provide an example for implicit parameters from companion objects (he only explains implicit conversion) but I think what this means is that a call to Container[A]#map should make all implicits from object Container available as implicit parameters, including my elementFactory. This assumption is supported by the fact that it is enough to provide the target type (no additional explicit imports!!) to get the implicit resolved.
The extra type parameter enables this behavior according to the specification of implicit resolution. Here is a summary from the FAQ answer to where do implicits come from (relevant parts bolded):
First look in current scope:
Implicits defined in current scope
Explicit imports
wildcard imports
Now look at associated types in:
Companion objects of a type (1)
Implicit scope of an argument’s type (2)
Implicit scope of type arguments (3)
Outer objects for nested types
Other dimensions
When you call map on a Mappable, you get the implicits from the implicit scope of its arguments, by (2). Since its argument is Factory in this case, that means that you also get the implicits from the implicit scope of all of type arguments of Factory, by (3). Next is the key: only when you add Source as a type parameter to Factory do you get all the implicits in the implicit scope of Container. Since the implicit scope of Container includes the implicits defined in its companion object (by (1)), this brings the desired implicit into scope without the imports you were using.
That's the syntactic reason, but there is a good semantic reason for this to be the desired behavior – unless the type is specified, the compiler would have no way of resolving the correct implicit when there may be conflicts. For example, if choosing between builder for a String or List[Char], the compiler needs to pick the correct one to enable "myFoo".map(_.toUpperCase) returning a String. If every implicit conversion were brought into scope all the time, this would be difficult or impossible. The above rules are meant to encompass an ordered list of what could possibly be relevant to the current scope in order to prevent this problem.
You can read more about implicits and implicit scope in the specification or in this great answer.
Here's why your other two solutions work: When you explicitly specify the return type of the call to map (in the version without the the Source parameter), then type inference comes into play, and the compiler can deduce that the parameter of interest That is Container, and therefore the implicit scope of Container comes into scope, including its companion object implicits. When you use an explicit import, then the desired implicit is in scope, trivially.
As for you meta-question, you can always click through the source code (or just checkout the repo), build minimal examples (like you have), and ask questions here. Using the REPL helps a lot to dealing with small examples like this.
The spec has a dedicated section on implicit parameter resolution. According to that section there are two sources for arguments to implicit parameters.
First, eligible are all identifiers x that can be accessed at the
point of the method call without a prefix and that denote an implicit
definition or an implicit parameter. An eligible identifier may thus
be a local name, or a member of an enclosing template, or it may be
have been made accessible without a prefix through an import clause.
So every name which is available without qualification and which is marked with the implicit keyword can be used as implicit parameter.
second eligible are also all implicit members of some object that
belongs to the implicit scope of the implicit parameter's type, T.
The implicit scope of a type T consists of all companion modules of
classes that are associated with the implicit parameter's type. Here,
we say a class C is associated with a type T if it is a base class of
some part of T.
Implicits defined in companion objects (a.k.a companion modules) of all parts of the types of an implicit parameter list are also available as parameter. So in the original example
def map[B, That](f: A => B)(implicit factory: Factory[Repr, B, That]): That
we will get implicits defined in the companion objects of Factory, Repr, B, and That. As Ben Reich has pointed out in his answer, this explains why the Repr type parameter is necessary to find the implicit parameter.
Here is why it works with the explicitly defined return type
val mapped: Container[Int] = c.map { x => 2 * x }
With the explicit return type definition, type inference selects Container[Int] for the That parameter in Factory[Repr, B, That]. Therefore, implicits defined in Container are then available as well.
I've stumbled upon a strange situation where having reflect.runtime.universe._ imported causes reflect.runtime.universe.RuntimeClass to be inferred where it seems Nothing would be more appropriate.
Consider this simple method and List:
import scala.reflect.ClassTag
def find[A : ClassTag](l: List[Any]): Option[A] =
l collectFirst { case a: A => a }
val list = List(1, "a", false)
I can use it to find the first element in the List of some type, and it works well, as expected.
scala> find[String](list)
res1: Option[String] = Some(a)
scala> find[Long](list)
res2: Option[Long] = None
If I do not supply the type parameter, then A is inferred as Nothing, so I get Option[Nothing], also as expected.
scala> find(list)
res3: Option[Nothing] = None
But, if I import scala.reflect.runtime.universe._ and again do not supply the type parameter, A is now inferred as reflect.runtime.universe.RuntimeClass instead of Nothing.
scala> find(list)
res4: Option[reflect.runtime.universe.RuntimeClass] = None
^ What?
This isn't a huge problem, since I can hardly imagine much of a use-case for the find method without supplying the type parameter manually, but why does this happen? The ClassTag seems partially to blame, as removing it once again causes Nothing to be inferred (though completely breaks the method due to erasure). What's going on here?
That looks like some completely unintended side effect of how the runtime Universe is designed internally.
The scala.reflect.runtime.universe has type scala.reflect.api.JavaUniverse.
By importing all its members, you import - in particular - a bunch of implicit ClassTag values defined in scala.reflect.api.ImplicitTags trait which is extended by the universe.
The ImplicitTags trait introduces about 90 different implicit ClassTag values. Among them, there is this one:
implicit val RuntimeClassTag: ClassTag[RuntimeClass]
It looks like the compiler likes it more than others since it decided to use it when inferring arbitrary ClassTag[A]. Why is that? This is because RuntimeClassTag is overridden in scala.reflect.api.JavaUniverse and implicit values defined in subclasses have priority over implicit values defined in superclasses (as specified somewhere in overloading resolution rules - SLS 6.26.3)
So, summarizing, the compiler inferred RuntimeClass for A with ClassTag context bound because it saw this thing in scope (introduced by wildcard import on universe):
trait JavaUniverse extends Universe { self =>
type RuntimeClass = java.lang.Class[_]
implicit val RuntimeClassTag: ClassTag[RuntimeClass] =
ClassTag[RuntimeClass](classOf[RuntimeClass])
...
}
#ghik's answer is correct, but I think there is one important thing to add. Normally, when we require an implicit ClassTag[A], we expect the compiler to generate the ClassTag for us. I was expecting ClassTag[Nothing] to be generated when omitting the explicit type parameter. But having the imported ClassTag[RuntimeClass] in scope appears to stop that from happening.
We can make this happen by simply introducing another implicit ClassTag without importing runtime.universe._.
scala> implicit val ct = classTag[Long]
ct: scala.reflect.ClassTag[Long] = Long
scala> find(list)
res8: Option[Long] = None
The compiler sees that I only have one implicit ClassTag[Long] in scope, so when I call find(list), the compiler assumes that must be what I wanted and A must be Long, because there is no other ClassTag available, and it will not generate another ClassTag for me.
In the precise context of my question then, I'm importing a boat-load of implicit ClassTags, and the compiler deems that ClassTag[RuntimeClass] is the most appropriate of those.
An implicit question to newcomers to Scala seems to be: where does the compiler look for implicits? I mean implicit because the question never seems to get fully formed, as if there weren't words for it. :-) For example, where do the values for integral below come from?
scala> import scala.math._
import scala.math._
scala> def foo[T](t: T)(implicit integral: Integral[T]) {println(integral)}
foo: [T](t: T)(implicit integral: scala.math.Integral[T])Unit
scala> foo(0)
scala.math.Numeric$IntIsIntegral$#3dbea611
scala> foo(0L)
scala.math.Numeric$LongIsIntegral$#48c610af
Another question that does follow up to those who decide to learn the answer to the first question is how does the compiler choose which implicit to use, in certain situations of apparent ambiguity (but that compile anyway)?
For instance, scala.Predef defines two conversions from String: one to WrappedString and another to StringOps. Both classes, however, share a lot of methods, so why doesn't Scala complain about ambiguity when, say, calling map?
Note: this question was inspired by this other question, in the hopes of stating the problem in a more general manner. The example was copied from there, because it is referred to in the answer.
Types of Implicits
Implicits in Scala refers to either a value that can be passed "automatically", so to speak, or a conversion from one type to another that is made automatically.
Implicit Conversion
Speaking very briefly about the latter type, if one calls a method m on an object o of a class C, and that class does not support method m, then Scala will look for an implicit conversion from C to something that does support m. A simple example would be the method map on String:
"abc".map(_.toInt)
String does not support the method map, but StringOps does, and there's an implicit conversion from String to StringOps available (see implicit def augmentString on Predef).
Implicit Parameters
The other kind of implicit is the implicit parameter. These are passed to method calls like any other parameter, but the compiler tries to fill them in automatically. If it can't, it will complain. One can pass these parameters explicitly, which is how one uses breakOut, for example (see question about breakOut, on a day you are feeling up for a challenge).
In this case, one has to declare the need for an implicit, such as the foo method declaration:
def foo[T](t: T)(implicit integral: Integral[T]) {println(integral)}
View Bounds
There's one situation where an implicit is both an implicit conversion and an implicit parameter. For example:
def getIndex[T, CC](seq: CC, value: T)(implicit conv: CC => Seq[T]) = seq.indexOf(value)
getIndex("abc", 'a')
The method getIndex can receive any object, as long as there is an implicit conversion available from its class to Seq[T]. Because of that, I can pass a String to getIndex, and it will work.
Behind the scenes, the compiler changes seq.IndexOf(value) to conv(seq).indexOf(value).
This is so useful that there is syntactic sugar to write them. Using this syntactic sugar, getIndex can be defined like this:
def getIndex[T, CC <% Seq[T]](seq: CC, value: T) = seq.indexOf(value)
This syntactic sugar is described as a view bound, akin to an upper bound (CC <: Seq[Int]) or a lower bound (T >: Null).
Context Bounds
Another common pattern in implicit parameters is the type class pattern. This pattern enables the provision of common interfaces to classes which did not declare them. It can both serve as a bridge pattern -- gaining separation of concerns -- and as an adapter pattern.
The Integral class you mentioned is a classic example of type class pattern. Another example on Scala's standard library is Ordering. There's a library that makes heavy use of this pattern, called Scalaz.
This is an example of its use:
def sum[T](list: List[T])(implicit integral: Integral[T]): T = {
import integral._ // get the implicits in question into scope
list.foldLeft(integral.zero)(_ + _)
}
There is also syntactic sugar for it, called a context bound, which is made less useful by the need to refer to the implicit. A straight conversion of that method looks like this:
def sum[T : Integral](list: List[T]): T = {
val integral = implicitly[Integral[T]]
import integral._ // get the implicits in question into scope
list.foldLeft(integral.zero)(_ + _)
}
Context bounds are more useful when you just need to pass them to other methods that use them. For example, the method sorted on Seq needs an implicit Ordering. To create a method reverseSort, one could write:
def reverseSort[T : Ordering](seq: Seq[T]) = seq.sorted.reverse
Because Ordering[T] was implicitly passed to reverseSort, it can then pass it implicitly to sorted.
Where do Implicits come from?
When the compiler sees the need for an implicit, either because you are calling a method which does not exist on the object's class, or because you are calling a method that requires an implicit parameter, it will search for an implicit that will fit the need.
This search obey certain rules that define which implicits are visible and which are not. The following table showing where the compiler will search for implicits was taken from an excellent presentation (timestamp 20:20) about implicits by Josh Suereth, which I heartily recommend to anyone wanting to improve their Scala knowledge. It has been complemented since then with feedback and updates.
The implicits available under number 1 below has precedence over the ones under number 2. Other than that, if there are several eligible arguments which match the implicit parameter’s type, a most specific one will be chosen using the rules of static overloading resolution (see Scala Specification §6.26.3). More detailed information can be found in a question I link to at the end of this answer.
First look in current scope
Implicits defined in current scope
Explicit imports
wildcard imports
Same scope in other files
Now look at associated types in
Companion objects of a type
Implicit scope of an argument's type (2.9.1)
Implicit scope of type arguments (2.8.0)
Outer objects for nested types
Other dimensions
Let's give some examples for them:
Implicits Defined in Current Scope
implicit val n: Int = 5
def add(x: Int)(implicit y: Int) = x + y
add(5) // takes n from the current scope
Explicit Imports
import scala.collection.JavaConversions.mapAsScalaMap
def env = System.getenv() // Java map
val term = env("TERM") // implicit conversion from Java Map to Scala Map
Wildcard Imports
def sum[T : Integral](list: List[T]): T = {
val integral = implicitly[Integral[T]]
import integral._ // get the implicits in question into scope
list.foldLeft(integral.zero)(_ + _)
}
Same Scope in Other Files
Edit: It seems this does not have a different precedence. If you have some example that demonstrates a precedence distinction, please make a comment. Otherwise, don't rely on this one.
This is like the first example, but assuming the implicit definition is in a different file than its usage. See also how package objects might be used in to bring in implicits.
Companion Objects of a Type
There are two object companions of note here. First, the object companion of the "source" type is looked into. For instance, inside the object Option there is an implicit conversion to Iterable, so one can call Iterable methods on Option, or pass Option to something expecting an Iterable. For example:
for {
x <- List(1, 2, 3)
y <- Some('x')
} yield (x, y)
That expression is translated by the compiler to
List(1, 2, 3).flatMap(x => Some('x').map(y => (x, y)))
However, List.flatMap expects a TraversableOnce, which Option is not. The compiler then looks inside Option's object companion and finds the conversion to Iterable, which is a TraversableOnce, making this expression correct.
Second, the companion object of the expected type:
List(1, 2, 3).sorted
The method sorted takes an implicit Ordering. In this case, it looks inside the object Ordering, companion to the class Ordering, and finds an implicit Ordering[Int] there.
Note that companion objects of super classes are also looked into. For example:
class A(val n: Int)
object A {
implicit def str(a: A) = "A: %d" format a.n
}
class B(val x: Int, y: Int) extends A(y)
val b = new B(5, 2)
val s: String = b // s == "A: 2"
This is how Scala found the implicit Numeric[Int] and Numeric[Long] in your question, by the way, as they are found inside Numeric, not Integral.
Implicit Scope of an Argument's Type
If you have a method with an argument type A, then the implicit scope of type A will also be considered. By "implicit scope" I mean that all these rules will be applied recursively -- for example, the companion object of A will be searched for implicits, as per the rule above.
Note that this does not mean the implicit scope of A will be searched for conversions of that parameter, but of the whole expression. For example:
class A(val n: Int) {
def +(other: A) = new A(n + other.n)
}
object A {
implicit def fromInt(n: Int) = new A(n)
}
// This becomes possible:
1 + new A(1)
// because it is converted into this:
A.fromInt(1) + new A(1)
This is available since Scala 2.9.1.
Implicit Scope of Type Arguments
This is required to make the type class pattern really work. Consider Ordering, for instance: It comes with some implicits in its companion object, but you can't add stuff to it. So how can you make an Ordering for your own class that is automatically found?
Let's start with the implementation:
class A(val n: Int)
object A {
implicit val ord = new Ordering[A] {
def compare(x: A, y: A) = implicitly[Ordering[Int]].compare(x.n, y.n)
}
}
So, consider what happens when you call
List(new A(5), new A(2)).sorted
As we saw, the method sorted expects an Ordering[A] (actually, it expects an Ordering[B], where B >: A). There isn't any such thing inside Ordering, and there is no "source" type on which to look. Obviously, it is finding it inside A, which is a type argument of Ordering.
This is also how various collection methods expecting CanBuildFrom work: the implicits are found inside companion objects to the type parameters of CanBuildFrom.
Note: Ordering is defined as trait Ordering[T], where T is a type parameter. Previously, I said that Scala looked inside type parameters, which doesn't make much sense. The implicit looked for above is Ordering[A], where A is an actual type, not type parameter: it is a type argument to Ordering. See section 7.2 of the Scala specification.
This is available since Scala 2.8.0.
Outer Objects for Nested Types
I haven't actually seen examples of this. I'd be grateful if someone could share one. The principle is simple:
class A(val n: Int) {
class B(val m: Int) { require(m < n) }
}
object A {
implicit def bToString(b: A#B) = "B: %d" format b.m
}
val a = new A(5)
val b = new a.B(3)
val s: String = b // s == "B: 3"
Other Dimensions
I'm pretty sure this was a joke, but this answer might not be up-to-date. So don't take this question as being the final arbiter of what is happening, and if you do noticed it has gotten out-of-date, please inform me so that I can fix it.
EDIT
Related questions of interest:
Context and view bounds
Chaining implicits
Scala: Implicit parameter resolution precedence
I wanted to find out the precedence of the implicit parameter resolution, not just where it looks for, so I wrote a blog post revisiting implicits without import tax (and implicit parameter precedence again after some feedback).
Here's the list:
1) implicits visible to current invocation scope via local declaration, imports, outer scope, inheritance, package object that are accessible without prefix.
2) implicit scope, which contains all sort of companion objects and package object that bear some relation to the implicit's type which we search for (i.e. package object of the type, companion object of the type itself, of its type constructor if any, of its parameters if any, and also of its supertype and supertraits).
If at either stage we find more than one implicit, static overloading rule is used to resolve it.
I have read the answer to my question about scala.math.Integral but I do not understand what happens when Integral[T] is passed as an implicit parameter. (I think I understand the implicit parameters concept in general).
Let's consider this function
import scala.math._
def foo[T](t: T)(implicit integral: Integral[T]) { println(integral) }
Now I call foo in REPL:
scala> foo(0)
scala.math.Numeric$IntIsIntegral$#581ea2
scala> foo(0L)
scala.math.Numeric$LongIsIntegral$#17fe89
How does the integral argument become scala.math.Numeric$IntIsIntegral and scala.math.Numeric$LongIsIntegral ?
The short answer is that Scala finds IntIsIntegral and LongIsIntegral inside the object Numeric, which is the companion object of the class Numeric, which is a super class of Integral.
Read on for the long answer.
Types of Implicits
Implicits in Scala refers to either a value that can be passed "automatically", so to speak, or a conversion from one type to another that is made automatically.
Implicit Conversion
Speaking very briefly about the latter type, if one calls a method m on an object o of a class C, and that class does not support method m, then Scala will look for an implicit conversion from C to something that does support m. A simple example would be the method map on String:
"abc".map(_.toInt)
String does not support the method map, but StringOps does, and there's an implicit conversion from String to StringOps available (see implicit def augmentString on Predef).
Implicit Parameters
The other kind of implicit is the implicit parameter. These are passed to method calls like any other parameter, but the compiler tries to fill them in automatically. If it can't, it will complain. One can pass these parameters explicitly, which is how one uses breakOut, for example (see question about breakOut, on a day you are feeling up for a challenge).
In this case, one has to declare the need for an implicit, such as the foo method declaration:
def foo[T](t: T)(implicit integral: Integral[T]) {println(integral)}
View Bounds
There's one situation where an implicit is both an implicit conversion and an implicit parameter. For example:
def getIndex[T, CC](seq: CC, value: T)(implicit conv: CC => Seq[T]) = seq.indexOf(value)
getIndex("abc", 'a')
The method getIndex can receive any object, as long as there is an implicit conversion available from its class to Seq[T]. Because of that, I can pass a String to getIndex, and it will work.
Behind the scenes, the compile changes seq.IndexOf(value) to conv(seq).indexOf(value).
This is so useful that there is a syntactic sugar to write them. Using this syntactic sugar, getIndex can be defined like this:
def getIndex[T, CC <% Seq[T]](seq: CC, value: T) = seq.indexOf(value)
This syntactic sugar is described as a view bound, akin to an upper bound (CC <: Seq[Int]) or a lower bound (T >: Null).
Please be aware that view bounds are deprecated from 2.11, you should avoid them.
Context Bounds
Another common pattern in implicit parameters is the type class pattern. This pattern enables the provision of common interfaces to classes which did not declare them. It can both serve as a bridge pattern -- gaining separation of concerns -- and as an adapter pattern.
The Integral class you mentioned is a classic example of type class pattern. Another example on Scala's standard library is Ordering. There's a library that makes heavy use of this pattern, called Scalaz.
This is an example of its use:
def sum[T](list: List[T])(implicit integral: Integral[T]): T = {
import integral._ // get the implicits in question into scope
list.foldLeft(integral.zero)(_ + _)
}
There is also a syntactic sugar for it, called a context bound, which is made less useful by the need to refer to the implicit. A straight conversion of that method looks like this:
def sum[T : Integral](list: List[T]): T = {
val integral = implicitly[Integral[T]]
import integral._ // get the implicits in question into scope
list.foldLeft(integral.zero)(_ + _)
}
Context bounds are more useful when you just need to pass them to other methods that use them. For example, the method sorted on Seq needs an implicit Ordering. To create a method reverseSort, one could write:
def reverseSort[T : Ordering](seq: Seq[T]) = seq.reverse.sorted
Because Ordering[T] was implicitly passed to reverseSort, it can then pass it implicitly to sorted.
Where do Implicits Come From?
When the compiler sees the need for an implicit, either because you are calling a method which does not exist on the object's class, or because you are calling a method that requires an implicit parameter, it will search for an implicit that will fit the need.
This search obey certain rules that define which implicits are visible and which are not. The following table showing where the compiler will search for implicits was taken from an excellent presentation about implicits by Josh Suereth, which I heartily recommend to anyone wanting to improve their Scala knowledge.
First look in current scope
Implicits defined in current scope
Explicit imports
wildcard imports
Same scope in other files
Now look at associated types in
Companion objects of a type
Companion objects of type parameters types
Outer objects for nested types
Other dimensions
Let's give examples for them.
Implicits Defined in Current Scope
implicit val n: Int = 5
def add(x: Int)(implicit y: Int) = x + y
add(5) // takes n from the current scope
Explicit Imports
import scala.collection.JavaConversions.mapAsScalaMap
def env = System.getenv() // Java map
val term = env("TERM") // implicit conversion from Java Map to Scala Map
Wildcard Imports
def sum[T : Integral](list: List[T]): T = {
val integral = implicitly[Integral[T]]
import integral._ // get the implicits in question into scope
list.foldLeft(integral.zero)(_ + _)
}
Same Scope in Other Files
This is like the first example, but assuming the implicit definition is in a different file than its usage. See also how package objects might be used in to bring in implicits.
Companion Objects of a Type
There are two object companions of note here. First, the object companion of the "source" type is looked into. For instance, inside the object Option there is an implicit conversion to Iterable, so one can call Iterable methods on Option, or pass Option to something expecting an Iterable. For example:
for {
x <- List(1, 2, 3)
y <- Some('x')
} yield, (x, y)
That expression is translated by the compile into
List(1, 2, 3).flatMap(x => Some('x').map(y => (x, y)))
However, List.flatMap expects a TraversableOnce, which Option is not. The compiler then looks inside Option's object companion and finds the conversion to Iterable, which is a TraversableOnce, making this expression correct.
Second, the companion object of the expected type:
List(1, 2, 3).sorted
The method sorted takes an implicit Ordering. In this case, it looks inside the object Ordering, companion to the class Ordering, and finds an implicit Ordering[Int] there.
Note that companion objects of super classes are also looked into. For example:
class A(val n: Int)
object A {
implicit def str(a: A) = "A: %d" format a.n
}
class B(val x: Int, y: Int) extends A(y)
val b = new B(5, 2)
val s: String = b // s == "A: 2"
This is how Scala found the implicit Numeric[Int] and Numeric[Long] in your question, by the way, as they are found inside Numeric, not Integral.
Companion Objects of Type Parameters Types
This is required to make the type class pattern really work. Consider Ordering, for instance... it comes with some implicits in its companion object, but you can't add stuff to it. So how can you make an Ordering for your own class that is automatically found?
Let's start with the implementation:
class A(val n: Int)
object A {
implicit val ord = new Ordering[A] {
def compare(x: A, y: A) = implicitly[Ordering[Int]].compare(x.n, y.n)
}
}
So, consider what happens when you call
List(new A(5), new A(2)).sorted
As we saw, the method sorted expects an Ordering[A] (actually, it expects an Ordering[B], where B >: A). There isn't any such thing inside Ordering, and there is no "source" type on which to look. Obviously, it is finding it inside A, which is a type parameter of Ordering.
This is also how various collection methods expecting CanBuildFrom work: the implicits are found inside companion objects to the type parameters of CanBuildFrom.
Outer Objects for Nested Types
I haven't actually seen examples of this. I'd be grateful if someone could share one. The principle is simple:
class A(val n: Int) {
class B(val m: Int) { require(m < n) }
}
object A {
implicit def bToString(b: A#B) = "B: %d" format b.m
}
val a = new A(5)
val b = new a.B(3)
val s: String = b // s == "B: 3"
Other Dimensions
I'm pretty sure this was a joke. I hope. :-)
EDIT
Related questions of interest:
Context and view bounds
Chaining implicits
The parameter is implicit, which means that the Scala compiler will look if it can find an implicit object somewhere that it can automatically fill in for the parameter.
When you pass in an Int, it's going to look for an implicit object that is an Integral[Int] and it finds it in scala.math.Numeric. You can look at the source code of scala.math.Numeric, where you will find this:
object Numeric {
// ...
trait IntIsIntegral extends Integral[Int] {
// ...
}
// This is the implicit object that the compiler finds
implicit object IntIsIntegral extends IntIsIntegral with Ordering.IntOrdering
}
Likewise, there is a different implicit object for Long that works the same way.