I am using scala 2.10.0-snapshot dated (20120522) and have the following Scala files:
this one defines the typeclass and a basic typeclass instance:
package com.netgents.typeclass.hole
case class Rabbit
trait Hole[A] {
def findHole(x: A): String
}
object Hole {
def apply[A: Hole] = implicitly[Hole[A]]
implicit val rabbitHoleInHole = new Hole[Rabbit] {
def findHole(x: Rabbit) = "Rabbit found the hole in Hole companion object"
}
}
this is the package object:
package com.netgents.typeclass
package object hole {
def findHole[A: Hole](x: A) = Hole[A].findHole(x)
implicit val rabbitHoleInHolePackage = new Hole[Rabbit] {
def findHole(x: Rabbit) = "Rabbit found the hole in Hole package object"
}
}
and here is the test:
package com.netgents.typeclass.hole
object Test extends App {
implicit val rabbitHoleInOuterTest = new Hole[Rabbit] {
def findHole(x: Rabbit) = "Rabbit found the hole in outer Test object"
}
{
implicit val rabbitHoleInInnerTest = new Hole[Rabbit] {
def findHole(x: Rabbit) = "Rabbit found the hole in inner Test object"
}
println(findHole(Rabbit()))
}
}
As you can see, Hole is a simple typeclass that defines a method which a Rabbit is trying to find. I am trying to figure out the implicit resolution rules on it.
with all four typeclass instances uncommented, scalac complains about ambiguities on rabbitHoleInHolePackage and rabbitHoleInHole. (Why?)
if I comment out rabbitHoleInHole, scalac compiles and I get back "Rabbit found the hole in Hole package object". (Shouldn't implicits in the local scope take precedence?)
if I then comment out rabbitHoleInHolePackage, scalac complains about ambiguities on rabbitHoleInOuterTest and rabbitHoleInInnerTest. (Why? In the article by eed3si9n, url listed below, he found implicits btw inner and outer scope can take different precedence.)
if I then comment out rabbitHoleInInnerTest, scalac compiles and I get back "Rabbit found the hole in outer Test object".
As you can see, the above behaviors do not follow the rules I've read on implicit resolution at all. I've only described a fraction of combinations you can do on commenting/uncommenting out instances and most of them are very strange indeed - and I haven't gotten into imports and subclasses yet.
I've read and watched presentation by suereth, stackoverflow answer by sobral, and a very elaborate revisit by eed3si9n, but I am still completely baffled.
Let's start with the implicits in the package object and the type class companion disabled:
package rabbit {
trait TC
object Test extends App {
implicit object testInstance1 extends TC { override def toString = "test1" }
{
implicit object testInstance2 extends TC { override def toString = "test2" }
println(implicitly[TC])
}
}
}
Scalac looks for any in scope implicits, finds testInstance1 and testInstance2. The fact that one is in a tighter scope is only relevant if they have the same name -- the normal rules of shadowing apply. We've chosen distinct names, and there neither implicit is more specific than the other, so an ambiguity is correctly reported.
Let's try another example, this time we'll play off an implicit in the local scope against one in the package object.
package rabbit {
object `package` {
implicit object packageInstance extends TC { override def toString = "package" }
}
trait TC
object Test extends App {
{
implicit object testInstance2 extends TC { override def toString = "test2" }
println(implicitly[TC])
}
}
}
What happens here? The first phase of the implicit search, as before, considers all implicits in scope at the call site. In this case, we have testInstance2 and packageInstance. These are ambiguous, but before reporting that error, the second phase kicks in, and searches the implicit scope of TC.
But what is in the implicit scope here? TC doesn't even have a companion object? We need to review the precise definition here, in 7.2 of the Scala Reference.
The implicit scope of a type T consists of all companion modules
(§5.4) of classes that are associated with the implicit parameter’s
type. Here, we say a class C is associated with a type T, if it
is a base class (§5.1.2) of some part of T.
The parts of a type T are:
if T is a compound type T1 with ... with Tn,
the union of the parts of T1, ..., Tn, as well as T itself,
if T is a parameterized type S[T1, ..., Tn], the union of the parts of S and
T1,...,Tn,
if T is a singleton type p.type, the parts of the type of p,
if T is a type projection S#U, the parts of S as well as T itself,
in all other cases, just T itself.
We're searching for rabbit.TC. From a type system perspective, this is a shorthand for: rabbit.type#TC, where rabbit.type is a type representing the package, as though it were a regular object. Invoking rule 4, gives us the parts TC and p.type.
So, what does that all mean? Simply, implicit members in the package object are part of the implicit scope, too!
In the example above, this gives us an unambiguous choice in the second phase of the implicit search.
The other examples can be explained in the same way.
In summary:
Implicit search proceeds in two phases. The usual rules of importing and shadowing determine a list of candidates.
implicit members in an enclosing package object may also be in scope, assuming you are using nested packages.
If there are more than one candidate, the rules of static overloading are used to see if there is a winner. An addiotnal a tiebreaker, the compiler prefers one implicit over another defined in a superclass of the first.
If the first phase fails, the implicit scope is consulted in the much same way. (A difference is that implicit members from different companions may have the same name without shadowing each other.)
Implicits in package objects from enclosing packages are also part of this implicit scope.
UPDATE
In Scala 2.9.2, the behaviour is different and wrong.
package rabbit {
trait TC
object Test extends App {
implicit object testInstance1 extends TC { override def toString = "test1" }
{
implicit object testInstance2 extends TC { override def toString = "test2" }
// wrongly considered non-ambiguous in 2.9.2. The sub-class rule
// incorrectly considers:
//
// isProperSubClassOrObject(value <local Test>, object Test)
// isProperSubClassOrObject(value <local Test>, {object Test}.linkedClassOfClass)
// isProperSubClassOrObject(value <local Test>, <none>)
// (value <local Test>) isSubClass <none>
// <notype> baseTypeIndex <none> >= 0
// 0 >= 0
// true
// true
// true
// true
//
// 2.10.x correctly reports the ambiguity, since the fix for
//
// https://issues.scala-lang.org/browse/SI-5354?focusedCommentId=57914#comment-57914
// https://github.com/scala/scala/commit/6975b4888d
//
println(implicitly[TC])
}
}
}
Related
The following is a simple example that tests the implicit feature of the scala 2.13 compiler:
object OverridingScope {
trait System {
trait Handler[T]
}
object Sys1 extends System {
object Handler {
implicit def handle1: Handler[Int] = new Handler[Int] {}
}
implicit def handle2: Handler[Long] = new Handler[Long] {}
}
}
According this article:
Where does Scala look for implicits?
The companion object of the trait Handler can be automatically imported as part of the implicit scope when type Handler[_] is used, the outer object Sys1 should be irrelevant. This means handle1 should be visible and handle2 should be hidden.
But the following test case shows the exact opposite:
class OverridingScope extends BaseSpec {
import OverridingScope._
it("implicits in companion is in scope") {
// val hh = implicitly[Sys1.Handler[Int]]
// Doesn't work
}
it("implicits in outer object is in scope") {
val hh = implicitly[Sys1.Handler[Long]]
// Should NOT work
}
}
Considering that Handler is both a class & object name, the fact that handle1 doesn't work looks highly suspicious. Also, I've checked the full list of implicit import:
First look in current scope
Implicits defined in current scope
Explicit imports
wildcard imports
Same scope in other files
Now look at associated types in
Companion objects of a type
Implicit scope of an argument's type (2.9.1)
Implicit scope of type arguments (2.8.0)
Outer objects for nested types
Other dimensions
None of the above can explain why handle2 is in the scope. What could be the reasons?
BTW: For those who dismissed implicit as a important scala feature: Implicit is the only way to implement type class, and if you want to implement it with minimal boilerplate, sometimes the above pattern can be the only choice.
Point 3 here explains that implicits/givens in prefixes of the type in question contribute to its implicit scope (in Scala 3 not if it's a package). So for the type Sys1.Handler[Long] implicits in Sys1 are in the implicit scope because Sys1 is a non-package prefix.
handle1 is not in the implicit scope because object Handler in object Sys1 is not the companion object of trait Handler in trait System.
I am learning Scala in order to use it for a project.
One thing I want to get a deeper understanding of is the type system, as it is something I have never used before in my other projects.
Suppose I have set up the following code:
// priority implicits
sealed trait Stringifier[T] {
def stringify(lst: List[T]): String
}
trait Int_Stringifier {
implicit object IntStringifier extends Stringifier[Int] {
def stringify(lst: List[Int]): String = lst.toString()
}
}
object Double_Stringifier extends Int_Stringifier {
implicit object DoubleStringifier extends Stringifier[Double] {
def stringify(lst: List[Double]): String = lst.toString()
}
}
import Double_Stringifier._
object Example extends App {
trait Animal[T0] {
def incrementAge(): Animal[T0]
}
case class Food[T0: Stringifier]() {
def getCalories = 100
}
case class Dog[T0: Stringifier]
(age: Int = 0, food: Food[T0] = Food()) extends Animal[String] {
def incrementAge(): Dog[T0] = this.copy(age = age + 1)
}
}
So in the example, there is a type error:
ambiguous implicit values:
[error] both object DoubleStringifier in object Double_Stringifier of type Double_Stringifier.DoubleStringifier.type
[error] and value evidence$2 of type Stringifier[T0]
[error] match expected type Stringifier[T0]
[error] (age: Int = 0, food: Food[T0] = Food()) extends Animal[String]
Ok fair enough. But if I remove the context bound, this code compiles. I.e. if I change the code for '''Dog''' to:
case class Dog[T0]
(age: Int = 0, food: Food[T0] = Food()) extends Animal[String] {
def incrementAge(): Dog[T0] = this.copy(age = age + 1)
}
Now I assumed that this would also not compile, because this type is more generic, so more ambiguous, but it does.
What is going on here? I understand that when I put the context bound, the compiler doesn't know whether it is a double or an int. But why then would an even more generic type compile? Surely if there is no context bound, I could potentially have a Dog[String] etc, which should also confuse the compiler.
From this answer: "A context bound describes an implicit value, instead of view bound's implicit conversion. It is used to declare that for some type A, there is an implicit value of type B[A] available"
Now I assumed that this would also not compile, because this type is more generic, so more ambiguous, but it does.
The ambiguity was between implicits. Both
Double_Stringifier.DoubleStringifier
and anonymous evidence of Dog[T0: Stringifier] (because class Dog[T0: Stringifier](...) is desugared to class Dog[T0](...)(implicit ev: Stringifier[T0])) were the candidates.
(Int_Stringifier#IntStringifier was irrelevant because it has lower priority).
Now you removed the context bound and only one candidate for implicit parameter in Food() remains, so there's no ambiguity. I can't see how the type being more generic is relevant. More generic doesn't mean more ambiguous. Either you have ambiguity between implicits or not.
Actually if you remove import but keep context bound the anonymous evidence is not seen in default values. So it counts for ambiguity but doesn't count when is alone :)
Scala 2.13.2, 2.13.3.
It seems to me (and if I'm wrong I'm hoping #DmytroMitin will correct me), the key to understanding this is with the default value supplied for the food parameter, which makes class Dog both a definition site, requiring an implicit be available at the call site, as well as a call site, requiring an implicit must be in scope at compile time.
The import earlier in the code supplies the implicit required for the Food() call site, but the Dog constructor requires an implicit, placed in ev, from its call site. Thus the ambiguity.
Here is a strange situation:
If I comment out the call to feed_usingExplicitTypeClassInstance below, then I get a compiler error.
Very puzzling. Any explanation ?
I mean, I comment out a function call (which returns no value) and then the code does not compile anymore ?
Should this be even possible at all in theory ? In any programming language ?
I mean I comment out something like println("hello") and then the code does not compile anymore ?
Of course it would be understandable if I would comment out a declaration or something, but a call to a function that does not return anything ?
object AnimalFeeder extends App {
def feed_usingExplicitTypeClassInstance[AnimalInstance]
(animalTypeClass: AnimalTypeClass[AnimalInstance])
(food: animalTypeClass.FoodThatAnimalLikes) =
{
animalTypeClass.feed(food)
}
def feed_usingImplicitTypeClassInstance[AnimalInstance, Food]
(food: Food)
(implicit animalTypeClass: AnimalTypeClass.Aux[Food,AnimalInstance]) =
{
animalTypeClass.feed(food)
}
// If I comment out this line, THEN !, I get an error !!!! How ???
feed_usingExplicitTypeClassInstance(AnimalTypeClass.CatInstance)(new CatFood())
feed_usingImplicitTypeClassInstance(new CatFood)
}
trait Food {
def eat(): Unit
}
trait AnimalTypeClass[AnimalInstance] {
type FoodThatAnimalLikes <: Food
def feed(f: FoodThatAnimalLikes) = f.eat()
}
object AnimalTypeClass {
type Aux[Food, Animal] = AnimalTypeClass[Animal] {
type FoodThatAnimalLikes = Food
}
implicit object CatInstance extends AnimalTypeClass[Cat] {
override type FoodThatAnimalLikes = CatFood
}
}
trait Cat
class CatFood extends Food {
override def eat(): Unit = println("meow")
}
This is the error:
Error:(23, 38) could not find implicit value for parameter animalTypeClass: AnimalTypeClass.Aux[CatFood,AnimalInstance]
feed_usingImplicitTypeClassInstance(new CatFood)
Error:(23, 38) not enough arguments for method feed_usingImplicitTypeClassInstance: (implicit animalTypeClass: AnimalTypeClass.Aux[CatFood,AnimalInstance])Unit.
Unspecified value parameter animalTypeClass.
feed_usingImplicitTypeClassInstance(new CatFood)
EDIT:
If I insert the line:
AnimalTypeClass.CatInstance
before:
feed_usingImplicitTypeClassInstance(new CatFood)
then the code compiles again, even if the line
feed_usingExplicitTypeClassInstance(AnimalTypeClass.CatInstance)(new CatFood())
is commented out.
This is a pretty well known issue, where implicits which appear after their usage in the same file and without an explicit type annotation are not found. For that reason it is strongly advised (and this will eventually be enforced) to give all non-local implicits an explicit type annotation. Unfortunately implicit objects are a bit tricky here, because they always act like implicit definitions without type annotation, and it is impossible to give them an explicit type... However last I checked this seemed to be fixed in Dotty for implicit objects.
See also, among others https://github.com/scala/bug/issues/8697
The reason that it does work when you uncomment a call to AnimalTypeClass.CatInstance in your code is that that reference will force the implicit object to be type checked earlier, so its type will be known before its implicit usage.
You have the definition of the implicit value in the same file after the usage of this value. It is not initialized when the compiler looks for an implicit value when you call feed_usingImplicitTypeClassInstance. Calling feed_usingExplicitTypeClassInstance with an explicit reference to this implicit value forces the implicit to initialize, and the compiler can use it in the implicit call.
Possible solutions:
Move the definition of the implicit value to another file.
If the implicit value is in the same file, move its definition above the place where you use it implicitly.
An easy thing to do in many languages but not in Scala is:
Define archetype 'Super', such that all implementations of 'Super' has to define a constructor 'create()'.
I found this constraint very important and is able to identify a lot of problems before runtime. However this feature is only partially enforced in Java (by defining an 'abstract' static method that always throws an error) and completely missing in Scala (companion object is completely detached from class and cannot be enforced in archetype).
is there a macro or tool that allows me to do this?
UPDATE Sorry my question was missing context and examples. Here is a formal use case in scala:
In project A, we define an interface that can be extended by all subprojects:
trait AbstractFoo {}
This interface should always have a default 0-parameter builder/constructor, so project A can initialize it on-demand, however, the implementation of each constructor is unknown to project A:
object AbstractFoo {
def default[T <: AbstractFoo: ClassTag](): T
}
So the problem becomes: How to rigorously define AbstractFoo, such that for all subprojects of A, any implementation(s) of AbstractFoo:
case class Foo(...) extends AbstractFoo
must satisfy:
'Foo' must have a 0-parameter builder/constructor defined (presumably in its companion object)
calling AbstractFoo.defaultFoo can invoke this 0-parameter builder/constructor
It should be noted that in an alternative conditions, a solution exists which is to define every companion object as an implicit type class:
trait FooBuilder[T <: AbstractFoo] {
def default(): T
}
object AbstractFoo {
implicit object Foo extends FooBuilder[Foo] {
def default() = {...}
}
def default[T <: AbstractFoo: FooBuilder](): T = {
implicitly[FooBuilder[T]].default
}
}
Such that if the implicit object is undefined the compiler will give an implicit not found error (my code snippet may have some syntax error, the idea is from http://www.cakesolutions.net/teamblogs/demystifying-implicits-and-typeclasses-in-scala)
Unfortunately it's not always convenient, because this subproject of A is usually unknown to project A. Yet the default implicit builder cannot be redefined, this makes every invocation of default() more covoluted.
I believe scala is a very extendable language, so there should be at least 1 way to enforce it whether if using macro, annotation or other metaprogramming techniques. Is my question clear enough now?
UPDATE2: I believe I found the solution after carefully study Scaladoc, there is a comment hidden in a corner:
if there are several eligible arguments which match the implicit parameter’s type, a most specific one will be chosen using the rules of static overloading resolution (see Scala Specification §6.26.4):
...
Implicit scope of type arguments (2.8.0)
...
So all I need is to write an implicit function in FooBuilder:
trait FooBuilder[T <: AbstractFoo] {
def default(): T
implicit def self = this
}
object Foo extends FooBuilder[Foo]
So everytime someone call:
default[Foo]
scala will refer to the scope of class Foo, which include object Foo, which contains the implicit value Foo, and eventually find the 0-parameter constructor.
I think this definition is better than defining it under object FooBuilder, since you can only define FooBuilder once, thus its not quite extendable. Would you agree with me? If so, could you please revise your answer so I can award you point?
I don't understand why an abstract class or even a Trait won't allow this to be done?
abstract class DefineCreate{
def create(): Unit
}
case class Foo(one: Int)
object Foo extends DefineCreate{
def create(): Unit = { Console.out.println("side-effect") }
}
Thus I force a user to make a create method on the object in question because all implementations of DefineCreate must do so in order to compile.
Update Following Comments
Well, without having to resort to macros and the like, you could achieve the same sort of thing with type classes:
trait Constructor[A]{
def create(): A
}
object Construct{
def create[A](implicit cr: Constructor[A]): A = cr.create()
}
Which doesn't explicitly force the companion object to sprout methods but it does force a user to make the type class if they want to use the Constructor.create[Foo] pattern.
Suppose we have implicit parameter lookup concerning only local scopes:
trait CanFoo[A] {
def foos(x: A): String
}
object Def {
implicit object ImportIntFoo extends CanFoo[Int] {
def foos(x: Int) = "ImportIntFoo:" + x.toString
}
}
object Main {
def test(): String = {
implicit object LocalIntFoo extends CanFoo[Int] {
def foos(x: Int) = "LocalIntFoo:" + x.toString
}
import Def._
foo(1)
}
def foo[A:CanFoo](x: A): String = implicitly[CanFoo[A]].foos(x)
}
In the above code, LocalIntFoo wins over ImportedIntFoo.
Could someone explain how it's considered more specific using "the rules of static overloading resolution (§6.26.3)"?
Edit:
The name binding precedence is a compelling argument, but there are several issues unresolved.
First, Scala Language Reference says:
If there are several eligible arguments which match the implicit parameter’s type, a most specific one will be chosen using the rules of static overloading resolution (§6.26.3).
Second, name binding precedence is about resolving a known identifier x to a particular member pkg.A.B.x in case there are several variable/method/object named x in the scope. ImportIntFoo and LocalIntFoo are not named the same.
Third, I can show that name binding precedence alone is not in play as follows:
trait CanFoo[A] {
def foos(x: A): String
}
object Def {
implicit object ImportIntFoo extends CanFoo[Int] {
def foos(x: Int) = "ImportIntFoo:" + x.toString
}
}
object Main {
def test(): String = {
implicit object LocalAnyFoo extends CanFoo[Any] {
def foos(x: Any) = "LocalAnyFoo:" + x.toString
}
// implicit object LocalIntFoo extends CanFoo[Int] {
// def foos(x: Int) = "LocalIntFoo:" + x.toString
// }
import Def._
foo(1)
}
def foo[A:CanFoo](x: A): String = implicitly[CanFoo[A]].foos(x)
}
println(Main.test)
Put this in test.scala and run scala test.scala, and it prints out ImportIntFoo:1.
This is because static overloading resolution (§6.26.3) says more specific type wins.
If we are pretending that all eligible implicit values are named the same, LocalAnyFoo should have masked ImportIntFoo.
Related:
Where does Scala look for implicits?
This is a great summary of implicit parameter resolution, but it quotes Josh's nescala presentation instead of the spec. His talk is what motivated me to look into this.
Compiler Implementation
rankImplicits
I wrote my own answer in the form of a blog post revisiting implicits without import tax.
Update: Furthermore, the comments from Martin Odersky in the above post revealed that the Scala 2.9.1's behavior of LocalIntFoo winning over ImportedIntFoo is in fact a bug. See implicit parameter precedence again.
1) implicits visible to current invocation scope via local declaration, imports, outer scope, inheritance, package object that are accessible without prefix.
2) implicit scope, which contains all sort of companion objects and package object that bear some relation to the implicit's type which we search for (i.e. package object of the type, companion object of the type itself, of its type constructor if any, of its parameters if any, and also of its supertype and supertraits).
If at either stage we find more than one implicit, static overloading rule is used to resolve it.
Update 2: When I asked Josh about Implicits without Import Tax, he explained to me that he was referring to name binding rules for implicits that are named exactly the same.
From http://www.scala-lang.org/docu/files/ScalaReference.pdf, Chapter 2:
Names in Scala identify types, values, methods, and classes which are
collectively called entities. Names are introduced by local definitions
and declarations (§4), inheritance (§5.1.3), import clauses (§4.7), or
package clauses (§9.2) which are collectively called bindings.
Bindings of different kinds have a precedence defined on them:
1. Definitions and declarations that are local, inherited, or made available by a package clause in the same compilation unit where the
definition occurs have highest precedence.
2. Explicit imports have next highest precedence.
3. Wildcard imports have next highest precedence.
4. Definitions made available by a package clause not in the compilation unit where the definition occurs have lowest precedence.
I may be mistaken, but the call to foo(1) is in the same compilation unit as LocalIntFoo, resulting in that conversion taking precedence over ImportedIntFoo.
Could someone explain how it's considered more specific using "the
rules of static overloading resolution (§6.26.3)"?
There's no method overload, so 6.26.3 is utterly irrelevant here.
Overload refers to multiple methods with the same name but different parameters being defined on the same class. For example, method f in the example 6.26.1 is overloaded:
class A extends B {}
def f(x: B, y: B) = . . .
def f(x: A, y: B) = . . .
val a: A
val b: B
Implicit parameter resolution precedence is a completely different rule, and one which has a question and answer already on Stack Overflow.