Arranging implicits precedence in Scala - scala

Consider the following (working) snippet that defines bidirectional implicit conversions between the DenseVector and Arrays:
import scala.reflect.ClassTag
import org.apache.spark.mllib.linalg.Vectors
import breeze.linalg.{DenseVector => BDV}
implicit def bdvToSeq[T](vect: BDV[T])(implicit ct: ClassTag[T]): Seq[T] = vect.toArray.toSeq
implicit def arrayToVect(darr: Array[Double]): BDV[Double]
= new BDV(darr)
implicit def bdvToArray[T](vect: BDV[T])(implicit ct: ClassTag[T]): Array[T] = vect.toArray
def add(v1: BDV[Double], v2: BDV[Double]) =
v1.zip(v2).map { x => x._1 + x._2}
Let's try it out:
scala> arr.allVisitableIndicesActive // Array to DenseVector implicit conversion
res4: Boolean = true
scala> val d = new BDV(Array(77.7, 88.8, 99.9))
d: breeze.linalg.DenseVector[Double] = DenseVector(77.7, 88.8, 99.9)
Now the other direction:
scala> def myArrMethod(arr: Array[Double]) = println(arr.mkString(","))
myArrMethod: (arr: Array[Double])Unit
scala> myArrMethod(d) // DenseVector to array conversion
77.7,88.8,99.9
The following also works: notice the darr.toSeq
def norm(darr: Array[Double]): Double = {
Math.sqrt(darr.toSeq.foldLeft(0.0) { case (sum, dval) => sum + Math.pow(dval, 2)})
}
However if we omit the .toSeq then we have implicits collision:
scala> def norm(darr: Array[Double]): Double = {
Math.sqrt(darr.foldLeft(0.0) { case (sum, dval) => sum + Math.pow(dval, 2)})
}
<console>:34: error: type mismatch;
found : darr.type (with underlying type Array[Double])
required: ?{def foldLeft(x$1: ? >: Double(0.0)): ?}
Note that implicit conversions are not applicable because they are ambiguous:
both method doubleArrayOps in object Predef of type (xs: Array[Double])scala.collection.mutable.ArrayOps[Double]
and method arrayToVect of type (darr: Array[Double])breeze.linalg.DenseVector[Double]
are possible conversion functions from darr.type to ?{def foldLeft(x$1: ? >: Double(0.0)): ?}
Math.sqrt(darr.foldLeft(0.0) { case (sum, dval) => sum + Math.pow(dval, 2)})
^
<console>:34: error: value foldLeft is not a member of Array[Double]
Math.sqrt(darr.foldLeft(0.0) { case (sum, dval) => sum + Math.pow(dval, 2)})
I would however prefer not to put the ".toSeq" explicitly.
So my question is: how to set the proper precedence so that the Predef.toSeq is the clear winner - always tried before the custom lower priority implicits defined above?

You can extract all your implicit conversions into object, and import them only where needed
trait BDVImplicits {
implicit def bdvToSeq[T](vect: BDV[T])(implicit ct: ClassTag[T]): Seq[T] =
vect.toArray.toSeq
implicit def arrayToVect(darr: Array[Double]): BDV[Double] =
new BDV(darr)
implicit def bdvToArray[T](vect: BDV[T])(implicit ct: ClassTag[T]): Array[T] =
vect.toArray
}
object BDVImplicits extends BDVImplicits
If you'll place your 'norm' method in Utils for example, and don't import implicit you'll be good. You'll just need to import BDVImplicits._ in class where you call your Util methods.
I think it's simplest workaround for this problem.

Related

How to correctly use ambiguous implicits for type negation in Scala

Ultimately what I want to do is provide one implementation of a type class for some specific type T and another implementation for all other types which are not T. I thought (perhaps incorrectly) that the easiest way to do this would be to try type negation via ambiguous implicits as described in this question. However, if I accidentally omit the implicit type class declaration, my code will still compile (should it?) but include bugs as only one of the implementations is used.
This is how the context bound is defined:
scala> trait NotAnInt[A]
defined trait NotAnInt
scala> implicit def everythingIsNotAnInt[A]: NotAnInt[A] = new NotAnInt[A] {}
everythingIsNotAnInt: [A]=> NotAnInt[A]
scala> implicit def intsAreInts1: NotAnInt[Int] = ???
intsAreInts1: NotAnInt[Int]
scala> implicit def intsAreInts2: NotAnInt[Int] = ???
intsAreInts2: NotAnInt[Int]
scala> implicit def nothingsAreInts1: NotAnInt[Nothing] = ???
nothingsAreInts1: NotAnInt[Nothing]
scala> implicit def nothingsAreInts2: NotAnInt[Nothing] = ???
nothingsAreInts2: NotAnInt[Nothing]
At this point NotAnInt[T] is summonable for all T except Int/Nothing:
scala> implicitly[NotAnInt[String]]
res3: NotAnInt[String] = $anon$1#1a24fe09
scala> implicitly[NotAnInt[Int]]
<console>:16: error: ambiguous implicit values:
both method intsAreInts1 of type => NotAnInt[Int]
and method intsAreInts2 of type => NotAnInt[Int]
match expected type NotAnInt[Int]
implicitly[NotAnInt[Int]]
^
scala> implicitly[NotAnInt[Nothing]]
<console>:18: error: ambiguous implicit values:
both method nothingsAreInts1 of type => NotAnInt[Nothing]
and method nothingsAreInts2 of type => NotAnInt[Nothing]
match expected type NotAnInt[Nothing]
implicitly[NotAnInt[Nothing]]
^
Now I have my NotAnInt context bound defined I can create my type class with its implementations:
scala> trait IntChecker[A] { def isInt(): Boolean }
defined trait IntChecker
scala> implicit val intIntChecker: IntChecker[Int] = new IntChecker[Int] { override def isInt = true }
intIntChecker: IntChecker[Int] = $anon$1#585dd35c
scala> implicit def otherIntChecker[A: NotAnInt]: IntChecker[A] = new IntChecker[A] { override def isInt = false }
otherIntChecker: [A](implicit evidence$1: NotAnInt[A])IntChecker[A]
This type class can be used as expected:
scala> def printIntStatus[T: IntChecker](t: T): Unit = { println(implicitly[IntChecker[T]].isInt()) }
printIntStatus: [T](t: T)(implicit evidence$1: IntChecker[T])Unit
scala> printIntStatus(3)
true
scala> printIntStatus("three")
false
However, the following also compiles:
scala> def printIntStatusWithBug[T](t: T): Unit = { println(implicitly[IntChecker[T]].isInt()) }
printIntStatusWithBug: [T](t: T)Unit
scala> printIntStatusWithBug(3)
false
scala> printIntStatusWithBug("three")
false
I would not expect this second function to compile as there should be no implicit IntChecker[T] available. I expect everythingIsNotAnInt is the cause of this problem but I can't think of a way around this.
I'm interested in why this approach fails as well as alternative methods on how to achieve the same thing. Thank you.
Consider the following alternative implementation (which uses Sabin's type inequalities)
trait =!=[A, B]
implicit def neq[A, B] : A =!= B = null
implicit def neqAmbig1[A] : A =!= A = null
implicit def neqAmbig2[A] : A =!= A = null
trait IntChecker[A] {
def isInt(): Boolean
}
object IntChecker {
import scala.reflect.ClassTag
implicit val intIntChecker: IntChecker[Int] = () => true
implicit def notIntIntChecker[T: ClassTag](implicit ev: T =!= Int): IntChecker[T] = () => false
}
def printIntStatus[T: IntChecker](t: T) = implicitly[IntChecker[T]].isInt()
import IntChecker._
printIntStatus(3)
printIntStatus("three")
which outputs
res0: Boolean = true
res1: Boolean = false
however the buggy implementation where we forget IntChecker bound
def printIntStatusWithBug[T](t: T) = implicitly[IntChecker[T]].isInt()
should not compile due to having T: ClassTag bound in
implicit def notIntIntChecker[T: ClassTag](implicit ev: T =!= Int)
giving compiler error
could not find implicit value for parameter e: IntChecker[T]
def printIntStatusWithBug[T](t: T) = implicitly[IntChecker[T]].isInt()
^

Scala chaining implicits conversions for Option[T]

I try to make implicit conversions chain that from Symbol -> A -> Option[A]. But fail to make it working with generic Option conversion, for example:
implicit def toInt(n: Symbol): Int = n.toString.length
implicit def symbolToString(n: Symbol): String = n.toString
implicit def toOptStr[T](b: T)(implicit fn: T ⇒ String): Option[String] = Option(b)
implicit def toOptInt[T](b: T)(implicit fn: T ⇒ Int): Option[Int] = Option(b)
And it works fine:
val c: Option[Int] = 'a25768xffff // returns Some(12)
val d: Option[String] = 'a2699 // returns Some('a2699)
But I have to explicitly define toOptStr[T](b: T): Option[String] and toOptInt[T](b: T)(implicit fn: T ⇒ Int): Option[Int]
What I want to achieve instead is to have only one generic toOptT conversion, which can convert to Option[T] provided there is implicit conversion from T=>V, something like following:
implicit def toInt(n: Symbol): Int = n.toString.length
implicit def symbolToString(n: Symbol): String = n.toString
implicit def toOptT[T,V](b: T)(implicit fn: T ⇒ V): Option[V] = Option(fn(b))
val c: Option[Int] = 'a25768xffff
val d: Option[String] = 'a2699
Unfortunately, 2 last lines give compilation error:
Error:(34, 93) type mismatch;
found : Symbol
required: Option[Int]
Error:(35, 96) type mismatch;
found : Symbol
required: Option[String]
Any help is very much appreciated.
Tried it on Scala 12.2.5. There is somewhat related question: Chain implicit conversion of collection, chaining implicits details FAQ: https://docs.scala-lang.org/tutorials/FAQ/chaining-implicits.html
Couldn't make it work on 2.12.5.
Here is one possible workaround:
implicit val toInt: Symbol => Int = _.toString.length
implicit val symbolToString: Symbol => String = _.toString
implicit class ToOptOps[S](s: S) {
def toOpt[T](implicit c: S => T): Option[T] = Option(c(s))
}
val c = 'a25768xffff.toOpt[Int]
val d = 'a2699.toOpt[String]
I'd advice you to use only typeclasses in combination with implicit classes to "pimp-the-interface" (adding new methods, as toOpt in this case). Here, one should actually replace the implicit c: S => T by a proper typeclass, to avoid unwanted collisions. The combination of type classes with "pimping" has proven to be robust. In contrast to that, implicit conversions seem just evil.

AnyRef equivalent for an AnyVal type

Given a type T, is there an idiomatic Scala way to describe the AnyRef equivalent of T (let's call it ARE[T]). For example,
ARE[T <: AnyRef] is T
ARE[T <: AnyVal] is the java.lang.* equivalent of T when one exists or a compilation error when it does not
The purpose of the question is to allow implementing many methods such as:
def foo[A](...): ARE[A]
while avoiding the naive def foo[A <: AnyRef](...): A + overloading for Boolean, Byte, Char, Double, Float, Int, Long and Short.
The standard way to implement a type computation like this is to create a typeclass:
sealed trait Box[T] {
type Out
def apply(t: T): Out
}
object Box {
type Aux[T, ARE] = Box[T] { type Out = ARE }
def make[T, ARE](f: T => ARE): Box.Aux[T, ARE] = new Box[T] {
type Out = ARE
def apply(t: T) = f(t)
}
implicit val int: Box.Aux[Int, java.lang.Integer] = make(Int.box)
implicit val long: Box.Aux[Long, java.lang.Long] = make(Long.box)
implicit val short: Box.Aux[Short, java.lang.Short] = make(Short.box)
implicit val byte: Box.Aux[Byte, java.lang.Byte] = make(Byte.box)
implicit val char: Box.Aux[Char, java.lang.Character] = make(Char.box)
implicit val float: Box.Aux[Float, java.lang.Float] = make(Float.box)
implicit val double: Box.Aux[Double, java.lang.Double] = make(Double.box)
implicit val boolean: Box.Aux[Boolean, java.lang.Boolean] = make(Boolean.box)
implicit val unit: Box.Aux[Unit, scala.runtime.BoxedUnit] = make(Unit.box)
implicit def anyRef[T <: AnyRef]: Box.Aux[T, T] = make(identity)
def box[T](t: T)(implicit are: Box[T]): are.Out = are(t)
}
This can be used like any other typeclass. For example, you can compute the type ARE with the help of Box.Aux in your own functions:
def box2[T, ARE](t: T)(implicit box: Box.Aux[T, ARE]): ARE = box(t)
Scala accepts the output of Box.box when AnyRef is expected:
scala> def foo[T <: AnyRef](anyRef: T): T = anyRef
foo: [T <: AnyRef](anyRef: T)T
scala> foo(10)
<console>:13: error: inferred type arguments [Int] do not conform to method foo's type parameter bounds [T <: AnyRef]
foo(10)
^
<console>:13: error: type mismatch;
found : Int(10)
required: T
foo(10)
^
scala> foo(Box.box(10))
res1: Box.int.Out = 10
And Scala also knows the exact ARE type returned from Box.box:
scala> def bar[T](t: T)(implicit ev: T =:= java.lang.Integer) = ev
bar: [T](t: T)(implicit ev: =:=[T,Integer])=:=[T,Integer]
scala> bar(10)
<console>:13: error: Cannot prove that Int =:= Integer.
bar(10)
^
scala> bar(Box.box(10))
res2: =:=[Box.int.Out,Integer] = <function1>

Scala: implicitly convert Option to Option of implicit class [duplicate]

I'm trying to write a function which re-uses the implicit conversions which I have for Object A -> Object B when they are wrapped in an Option in a generic way so that Option[A] -> Option[B] conversions also work.
What I've come up with is:
implicit def fromOptionToOption[A, B](from: Option[A])(implicit conversion: (A) => B): Option[B] = from.map(conversion(_))
This works when I assign a Some(..) to a value but not when I assign an Option val; see the following console output:
scala> trait T
defined trait T
scala> case class Foo(i: Int) extends T
defined class Foo
scala> case class Bar(i: Int) extends T
defined class Bar
scala> implicit def fromFooToBar(f: Foo):Bar = Bar(f.i)
fromFooToBar: (f: Foo)Bar
scala> implicit def fromBarToFoo(b: Bar):Foo = Foo(b.i)
fromBarToFoo: (b: Bar)Foo
scala> implicit def fromOptionToOption[A, B](from: Option[A])(implicit conversion: (A) => B): Option[B] = from.map(conversion(_))
fromOptionToOption: [A, B](from: Option[A])(implicit conversion: (A) => B)Option[B]
scala> val foo: Option[Foo] = Some(Bar(1))
foo: Option[Foo] = Some(Foo(1))
// THIS WORKS as expected
scala> val fooOpt = Some(Foo(4))
fooOpt: Some[Foo] = Some(Foo(4))
scala> val barOpt2: Option[Bar] = fooOpt
<console>:16: error: type mismatch;
found : Some[Foo]
required: Option[Bar]
val barOpt2: Option[Bar] = fooOpt
^
//THIS FAILS.
I don't really see the difference between the first and second conversion. Somehow it doesn't invoke the implicit conversion in the latter. I guess it has something to do with the type system, but I can't see how just yet. Any ideas?
-Albert
(I'm on scala 2.9.1)
Here's clue:
scala> val fooOpt: Option[Bar] = Option(Foo(1))
fooOpt: Option[Bar] = Some(Bar(1))
And another:
scala> implicit def foobar(x: String): Int = augmentString(x).toInt
foobar: (x: String)Int
scala> val y: Option[String] = Option(1)
y: Option[String] = Some(1)
scala> val y: Option[Int] = Option("1")
y: Option[Int] = Some(1)
Looks like a legitimately odd bug. I'd pop open a smaller test case and open an issue (or search for one in JIRA).
As an aside:
You could use some category theory to handle lots of different types of "Option-ish" things.
package object fun {
trait Functor[Container[_]] {
def fmap[A,B](x: Container[A], f: A => B): Container[B]
}
object Functor {
implicit object optionFunctor extends Functor[Option] {
override def fmap[A,B](x: Option[A], f: A => B): Option[B] = x map f
}
// Note: With some CanBuildFrom magic, we can support Traversables here.
}
implicit def liftConversion[F[_], A, B](x: F[A])(implicit f: A => B, functor: Functor[F]): F[B] =
functor.fmap(x,f)
}
That's a bit more advanced, as you're mapping some category theory FP onto the problem, but it's a more general solution to lift implicit conversations into containers as needed. Notice how they chain by using one implicit conversation method that takes a more limited implicit argument.
ALSO, this should make the examples work:
scala> val tmp = Option(Foo(1))
tmp: Option[Foo] = Some(Foo(1))
scala> val y: Option[Bar] = tmp
y: Option[Bar] = Some(Bar(1))
And make your usage of Some more dangerous:
scala> val tmp = Some(Foo(1))
tmp: Some[Foo] = Some(Foo(1))
scala> val y: Option[Bar] = tmp
<console>:25: error: could not find implicit value for parameter functor: fun.Functor[Some]
val y: Option[Bar] = tmp
^
That's telling you that variance is critical, and interacts with implicits. My guess is you ran into a very rare, probably hard to fix bug that can be avoided using other techniques.
You might not be aware of it, but there's a flag for that: -Xlog-implicits. And this is what it says:
scala> val barOpt2: Option[Bar] = fooOpt
fromOptionToOption is not a valid implicit value for Some[Foo] => Option[Bar] because:
incompatible: (from: Option[Foo])(implicit conversion: Foo => B)Option[B] does not match expected type Some[Foo] => Option[Bar]
<console>:16: error: type mismatch;
found : Some[Foo]
required: Option[Bar]
val barOpt2: Option[Bar] = fooOpt
^
And there you go -- it doesn't know what type B must be. 0__ mentioned that this problem doesn't happen with invariant collections, and that makes some sense. In invariant collections, B must be exactly Bar, while for covariant collections it could be any subtype of Bar.
So, why does val foo: Option[Foo] = Some(Bar(1)) work? Well, there's a flag for that too... -Ytyper-debug. Not for the weak, however, given the extreme verbosity.
I waddled through anyway, comparing what happens in both cases, and the answer is rather simple... it's not the Option that is being converted in that case, but Bar! Remember, you declared an implicit conversion from Bar => Foo, so it is applying that conversion before passing the result to Some!
It doesn't work because the Scala Language Specification defines view as follows:
Implicit parameters and methods can also define implicit conversions called views. A view from type S to type T is defined by an implicit value which has function type S=>T or (=>S)=>T or by a method convertible to a value of that type.
fromOptionToOption doesn't conform to the three categories since it takes an implicit parameter. Compiler doesn't seem to find converter with both destination and source having generic type.
Defining a view from Option[Foo] to Option[Bar] works as expected.
trait T
case class Foo(i: Int) extends T
case class Bar(i: Int) extends T
object Main {
implicit def fromFooToBar(f: Foo):Bar = Bar(f.i)
implicit def fromBarToFoo(b: Bar):Foo = Foo(b.i)
// implicit def fromOptionToOption[A, B](from: Option[A])(implicit conversion: (A) => B): Option[B] =
// from.map(conversion(_))
implicit def fromOptionFooToOptionBar(o: Option[Foo]): Option[Bar] = o map { foo => foo }
def test(): Option[Bar] = {
val fooOpt = Some(Foo(4))
val barOpt2: Option[Bar] = fooOpt
barOpt2
}
}
println(Main.test)
Running this prints out:
$ scala so.scala
Some(Bar(4))
However, all is not lost. It's not as nice as general Option to Option, but we can do something like anything that can turn into Bar to Option[Bar] by view bound.
trait T
case class Foo(i: Int) extends T
case class Bar(i: Int) extends T
object Main {
implicit def fromFooToBar(f: Foo):Bar = Bar(f.i)
implicit def fromBarToFoo(b: Bar):Foo = Foo(b.i)
implicit def fromOptionToOptionBar[A <% Bar](from: Option[A]): Option[Bar] =
from map { foo => foo }
def test(): Option[Bar] = {
val fooOpt = Some(Foo(4))
val barOpt2: Option[Bar] = fooOpt
barOpt2
}
}
println(Main.test)
Here's another workaround that can be used for general Option to Option but requires extra .convert call:
trait T
case class Foo(i: Int) extends T
case class Bar(i: Int) extends T
case class Converter[A](x: Option[A]) {
def convert[B](implicit ev: Function1[A, B]): Option[B] = x map { a: A => ev(a) }
}
object Main {
implicit def optionToConverter[A](x: Option[A]) = Converter(x)
implicit def fooToBar(x: Foo) = Bar(x.i)
def test(): Option[Bar] = {
val fooOpt = Some(Foo(4))
val barOpt: Option[Bar] = fooOpt.convert
barOpt
}
}
println(Main.test)
Indeed it's a very strange problem. I tried to use another type than Option, and it turns out that the problem is that Option is covariant in its type parameter. This works all:
case class A[B](value: B) // invariant in B
case class X()
case class Y()
implicit def xtoy(x: X): Y = Y()
implicit def ytox(x: Y): X = X()
implicit def movea[U, V](from: A[U])(implicit view: U => V): A[V] = A[V](from.value)
def test(a: A[Y]) = "ok"
test(A(X())) // (1)
val f = A(X())
test(f) // (2)
But if instead I define A as
case class A[+B](value: B) // covariant in B
The case (2) fails. Case (1) always succeeds, because Scala already converts X to Y before wrapping it in an A.
Now that we know the problem source, you need to wait for a type guru to explain why this is actually a problem... The conversion is still valid, you see:
askForY(movea(f)) // succeeds, even with A[+B]
I improved #jseureth answer and added support for Traversable:
trait Mappable[A, B, C[_]] {
def apply(f: A => B): C[B]
}
package object app {
implicit class OptionMappable[A, B, C[X] <: Option[X]](option: C[A]) extends Mappable[A, B, Option] {
override def apply(f: A => B): Option[B] = option.map(f)
}
implicit class TraversableMappable[A, B, C[X] <: Traversable[X]](traversable: C[A])
(implicit cbf: CanBuildFrom[C[A], B, C[B]]) extends Mappable[A, B, C] {
override def apply(f: A => B): C[B] = {
val builder = cbf(traversable)
builder.sizeHint(traversable)
builder ++= traversable.map(f)
builder.result()
}
}
implicit def liftConversion[C[_], A, B](x: C[A])
(implicit f: A => B, m: C[A] => Mappable[A, B, C]): C[B] = m(x)(f)
}
Now you can implicitly convert options and traversables:
implicit def f(i: Int): String = s"$i"
val a: Option[String] = Some(1)
val b: Seq[String] = Seq(1, 2, 3)

diverging implicit expansion

Can someone please explain to me why I get a 'diverging implicit expansion error' here? I think it has to do with the type synonym type MyIdType = String but I am not sure why.
import org.scalacheck.Arbitrary
import org.scalacheck.Arbitrary._
import org.scalacheck.Gen
def arbitraryCaseClass[A,C](f: A => C)(implicit t: Arbitrary[A]): Arbitrary[C] = Arbitrary(for(v <- arbitrary[A]) yield f(v))
type MyIdType = String
implicit val arbMyIdType: Arbitrary[MyIdType] = Arbitrary(Gen.identifier)
case class Foo(s: MyIdType, t: Int)
implicit def arbA = arbitraryCaseClass((Foo.apply _).tupled)
val foo = arbitrary[Foo].sample
error:
Error:(13, 40) diverging implicit expansion for type org.scalacheck.Arbitrary[(A$A6.this.MyIdType, Int)]
starting with method arbTuple2 in trait ArbitraryArities
implicit def arbA = arbitraryCaseClass((Foo.apply _).tupled)