I try to make implicit conversions chain that from Symbol -> A -> Option[A]. But fail to make it working with generic Option conversion, for example:
implicit def toInt(n: Symbol): Int = n.toString.length
implicit def symbolToString(n: Symbol): String = n.toString
implicit def toOptStr[T](b: T)(implicit fn: T ⇒ String): Option[String] = Option(b)
implicit def toOptInt[T](b: T)(implicit fn: T ⇒ Int): Option[Int] = Option(b)
And it works fine:
val c: Option[Int] = 'a25768xffff // returns Some(12)
val d: Option[String] = 'a2699 // returns Some('a2699)
But I have to explicitly define toOptStr[T](b: T): Option[String] and toOptInt[T](b: T)(implicit fn: T ⇒ Int): Option[Int]
What I want to achieve instead is to have only one generic toOptT conversion, which can convert to Option[T] provided there is implicit conversion from T=>V, something like following:
implicit def toInt(n: Symbol): Int = n.toString.length
implicit def symbolToString(n: Symbol): String = n.toString
implicit def toOptT[T,V](b: T)(implicit fn: T ⇒ V): Option[V] = Option(fn(b))
val c: Option[Int] = 'a25768xffff
val d: Option[String] = 'a2699
Unfortunately, 2 last lines give compilation error:
Error:(34, 93) type mismatch;
found : Symbol
required: Option[Int]
Error:(35, 96) type mismatch;
found : Symbol
required: Option[String]
Any help is very much appreciated.
Tried it on Scala 12.2.5. There is somewhat related question: Chain implicit conversion of collection, chaining implicits details FAQ: https://docs.scala-lang.org/tutorials/FAQ/chaining-implicits.html
Couldn't make it work on 2.12.5.
Here is one possible workaround:
implicit val toInt: Symbol => Int = _.toString.length
implicit val symbolToString: Symbol => String = _.toString
implicit class ToOptOps[S](s: S) {
def toOpt[T](implicit c: S => T): Option[T] = Option(c(s))
}
val c = 'a25768xffff.toOpt[Int]
val d = 'a2699.toOpt[String]
I'd advice you to use only typeclasses in combination with implicit classes to "pimp-the-interface" (adding new methods, as toOpt in this case). Here, one should actually replace the implicit c: S => T by a proper typeclass, to avoid unwanted collisions. The combination of type classes with "pimping" has proven to be robust. In contrast to that, implicit conversions seem just evil.
Related
So I would like to use a generic return type and be able to use the info of that type within the function. Not sure this is possible but here is what I would like:
def getStuff[A](a: MyObj, b: String): Option[A] = {
// do some stuff
A match {
case String => Some(a.getString(b))
case Integer => Some(a.getInt(b))
...
case _ => None
}
}
However, as you know, A match is not a possibility. Any ideas on how I could achieve this ?
This is a classic case for using a typeclass:
trait StuffGetter[T] { // typeclass
def get(obj: MyObj, s: String): Option[T]
}
implicit val stringGetter = new StuffGetter[String] {
def get(o: MyObj, s: String): Option[String] = ???
}
implicit val intGetter = new StuffGetter[Int] {
def get(o: MyObj, s: String): Option[Int] = ???
}
def getStuff[A](a: MyObj, b: String)(implicit ev: StuffGetter[A]): Option[A] =
ev.get(a, b)
val stuff0 = getStuff[String](obj, "Hello") // calls get on stringGetter
val stuff1 = getStuff[Int](obj, "World") // call get on intGetter
val stuff2 = getStuff[Boolean](obj, "!") // Compile-time error
The StuffGetter trait defines the operations that you want to perform on the generic type, and each implicit value of that trait provides the implementation for a specific type. (For a custom type these are typically place in the companion object for the type; the compiler will look there for them)
When getStuff is called the compiler will look for an implicit instance of StuffGetter with the matching type. This will fail if no such instance exists, otherwise it will be passed in the ev parameter.
The advantage of this is that the "match" is done at compile time and unsupported types are also detected at compile time.
Conceptually we can differentiate between pattern matching at run-time which looks something like this
def getStuff[A](...) =
A match {
...
}
and pattern matching at compile-time which looks something like this
def getStuff[A](...)(implicit ev: Foo[A]) = {
ev.bar(...)
}
Key concept to understand is that types do not exists at run-time because they get "erased" after compilation so there is not enough information to pattern match on types once the program is running. However at compile-time, that is before the program runs, types do exist and Scala provides means to ask the compiler to effectively pattern match on them via implicit/givens mechanism which looks something like so
// type class requirements for type A
trait StringConverter[A] {
def getOptValue(b: String): Option[A]
}
// evidence which types satisfy the type class
implicit val intStringConverter: StringConverter[Int] = (b: String) => b.toIntOption
implicit val strStringConverter: StringConverter[String] = (b: String) => Some(b)
implicit def anyStringConverter[A]: StringConverter[A] = (b: String) => None
// compile-time pattern matching on type A
def getStuff[A](b: String)(implicit ev: StringConverter[A]): Option[A] = {
ev.getOptValue(b)
}
getStuff[Int]("3") // : Option[Int] = Some(value = 3)
getStuff[String]("3") // : Option[String] = Some(value = "3")
getStuff[Double]("3") // : Option[Double] = None
This compile-time pattern matching is called type class pattern.
Understanding the distinction between types and classes is one of the fundamental concepts in Scala https://docs.scala-lang.org/tutorials/FAQ/index.html#whats-the-difference-between-types-and-classes and gorking it will help understand how to write type classes.
Using custom typeclass similar to Getter:
trait KeyedGetter[S, K, A]:
def get(s: S, key: K): Option[A]
case class MyObj(ints: Map[String, Int], strs: Map[String, String])
object MyObj:
given KeyedGetter[MyObj, String, Int] with
def get(m: MyObj, k: String) = m.ints.get(k)
given KeyedGetter[MyObj, String, String] with
def get(m: MyObj, k: String) = m.strs.get(k)
def getStuff[A](m: MyObj, key: String)(using g: KeyedGetter[MyObj, String, A]): Option[A] =
g.get(m, key)
Using class tags:
case class MyObj(ints: Map[String, Int], strs: Map[String, String])
import reflect._
def getStuff[A](m: MyObj, key: String)(using ct: ClassTag[A]): Option[A] = (ct match
case _ if ct == classTag[String] => m.strs.get(key)
case _ if ct == classTag[Int] => m.ints.get(key)
case _ => None
).asInstanceOf[Option[A]]
If the erased types are insufficient, for a similar approach with type tags see this answer (and ignore the rest).
Compiler does know type parameter T is a String inside method body because of generalised type constraint T =:= String in the following example
scala> def f[A](a: A)(implicit ev: A =:= String) = a.toUpperCase
def f[A](a: A)(implicit ev: A =:= String): String
however despite having this knowledge it seems it is not fully applied. For example consider how we cannot reverse a String
scala> def f[A <: String](a: A) = a.reverse
def f[A <: String](a: A): String
scala> def f[A](a: A)(implicit ev: A <:< String) = a.reverse
^
error: value reverse is not a member of type parameter A
or how we cannot assign a String value to A despite compiler knowing A is a String
scala> def f[A](a: String)(implicit ev: A =:= String) = { var x: A = a }
^
error: type mismatch;
found : a.type (with underlying type String)
required: A
What explains this behaviour, that is, why are generalised type constraints not fully effective inside method body?
In the tradition of harvesting Luis' points buried in comments, the beauty of syntactic sugar of =:= and <:< might mask the fact that there is nothing special about them. They are similar to
scala> def f[A](a: A)(implicit ev: A => String) = a.toUpperCase
def f[A](a: A)(implicit ev: A => String): String
scala> def f[A](a: A)(implicit ev: A => String) = ev.apply(a).reverse
def f[A](a: A)(implicit ev: A => String): String
where compiler utilises evidence as implicit conversion where necessary
def f[A](a: A)(implicit ev: A => String): String =
ev.apply(a).toUpperCase
def f[A](a: A)(implicit ev: A => String): String =
Predef.augmentString(ev.apply(a)).reverse
| |
| 1st conversion
2nd conversion
In the case of reverse we cannot simply write a.reverse as there are two chained conversions necessary before we can call reverse and Scala cannot do that automatically so we need to be explicit about one step in the chain.
Viewing A =:= String as A => String we see that type parameter A remains unspecialised inside the method body, hence for example var x: A = a cannot work because A is not bound to any special type.
Given a type T, is there an idiomatic Scala way to describe the AnyRef equivalent of T (let's call it ARE[T]). For example,
ARE[T <: AnyRef] is T
ARE[T <: AnyVal] is the java.lang.* equivalent of T when one exists or a compilation error when it does not
The purpose of the question is to allow implementing many methods such as:
def foo[A](...): ARE[A]
while avoiding the naive def foo[A <: AnyRef](...): A + overloading for Boolean, Byte, Char, Double, Float, Int, Long and Short.
The standard way to implement a type computation like this is to create a typeclass:
sealed trait Box[T] {
type Out
def apply(t: T): Out
}
object Box {
type Aux[T, ARE] = Box[T] { type Out = ARE }
def make[T, ARE](f: T => ARE): Box.Aux[T, ARE] = new Box[T] {
type Out = ARE
def apply(t: T) = f(t)
}
implicit val int: Box.Aux[Int, java.lang.Integer] = make(Int.box)
implicit val long: Box.Aux[Long, java.lang.Long] = make(Long.box)
implicit val short: Box.Aux[Short, java.lang.Short] = make(Short.box)
implicit val byte: Box.Aux[Byte, java.lang.Byte] = make(Byte.box)
implicit val char: Box.Aux[Char, java.lang.Character] = make(Char.box)
implicit val float: Box.Aux[Float, java.lang.Float] = make(Float.box)
implicit val double: Box.Aux[Double, java.lang.Double] = make(Double.box)
implicit val boolean: Box.Aux[Boolean, java.lang.Boolean] = make(Boolean.box)
implicit val unit: Box.Aux[Unit, scala.runtime.BoxedUnit] = make(Unit.box)
implicit def anyRef[T <: AnyRef]: Box.Aux[T, T] = make(identity)
def box[T](t: T)(implicit are: Box[T]): are.Out = are(t)
}
This can be used like any other typeclass. For example, you can compute the type ARE with the help of Box.Aux in your own functions:
def box2[T, ARE](t: T)(implicit box: Box.Aux[T, ARE]): ARE = box(t)
Scala accepts the output of Box.box when AnyRef is expected:
scala> def foo[T <: AnyRef](anyRef: T): T = anyRef
foo: [T <: AnyRef](anyRef: T)T
scala> foo(10)
<console>:13: error: inferred type arguments [Int] do not conform to method foo's type parameter bounds [T <: AnyRef]
foo(10)
^
<console>:13: error: type mismatch;
found : Int(10)
required: T
foo(10)
^
scala> foo(Box.box(10))
res1: Box.int.Out = 10
And Scala also knows the exact ARE type returned from Box.box:
scala> def bar[T](t: T)(implicit ev: T =:= java.lang.Integer) = ev
bar: [T](t: T)(implicit ev: =:=[T,Integer])=:=[T,Integer]
scala> bar(10)
<console>:13: error: Cannot prove that Int =:= Integer.
bar(10)
^
scala> bar(Box.box(10))
res2: =:=[Box.int.Out,Integer] = <function1>
Using the Scala REPL, I've defined a function that takes an Int as its first parameter and a function with this signature Int => Int as the second implicit parameter:
scala> def doer(i: Int)(implicit a: Int => Int): Int = a(i)
doer: (i: Int)(implicit a: Int => Int)Int
Why does running this function without providing the implicit parameter work?
scala> doer(4)
res1: Int = 4
Where does the implicit Int to Int function come from?
The REPL reports that there are no implicits defined:
scala> :impl
No implicits have been imported other than those in Predef.
Predef contains an implicit evidence that one type is a subtype of another: A <:< B. Every type is a subtype of itself so implicitly[Int <:< Int] works. This <:< class extends function A => B. So that's why implicitly[Int => Int] also works.
Int and java.lang.Integer however are different things with no subtype relation whatsoever, so these int2Integer implicits have nothing to do with it.
If you have a REPL of a recent Scala version you can type
scala> doer(4) //print
And press the tab key instead of enter. It will show you the desugared version doer(4)(scala.Predef.$conforms[Int]) of your code with all implicits filled in explicitly.
There are bunch of implicits defined in scala.Predef which are accessible in all Scala compilation units without explicit qualification, and one of them is
implicit def int2Integer(x: Int): java.lang.Integer = x.asInstanceOf[java.lang.Integer]
If you check the Predef source code it is possible to see a lot of implicit functions. The compiler will just pick up a compatible one.
Some examples:
implicit def booleanArrayOps(xs: Array[Boolean]): ArrayOps[Boolean] = new ArrayOps.ofBoolean(xs)
implicit def byteArrayOps(xs: Array[Byte]): ArrayOps[Byte] = new ArrayOps.ofByte(xs)
implicit def charArrayOps(xs: Array[Char]): ArrayOps[Char] = new ArrayOps.ofChar(xs)
implicit def doubleArrayOps(xs: Array[Double]): ArrayOps[Double] = new ArrayOps.ofDouble(xs)
implicit def floatArrayOps(xs: Array[Float]): ArrayOps[Float] = new ArrayOps.ofFloat(xs)
implicit def intArrayOps(xs: Array[Int]): ArrayOps[Int] = new ArrayOps.ofInt(xs)
implicit def longArrayOps(xs: Array[Long]): ArrayOps[Long] = new ArrayOps.ofLong(xs)
implicit def refArrayOps[T <: AnyRef](xs: Array[T]): ArrayOps[T] = new ArrayOps.ofRef[T](xs)
implicit def shortArrayOps(xs: Array[Short]): ArrayOps[Short] = new ArrayOps.ofShort(xs)
implicit def unitArrayOps(xs: Array[Unit]): ArrayOps[Unit] = new ArrayOps.ofUnit(xs)
// "Autoboxing" and "Autounboxing" ---------------------------------------------------
implicit def byte2Byte(x: Byte) = java.lang.Byte.valueOf(x)
implicit def short2Short(x: Short) = java.lang.Short.valueOf(x)
implicit def char2Character(x: Char) = java.lang.Character.valueOf(x)
implicit def int2Integer(x: Int) = java.lang.Integer.valueOf(x)
implicit def long2Long(x: Long) = java.lang.Long.valueOf(x)
implicit def float2Float(x: Float) = java.lang.Float.valueOf(x)
implicit def double2Double(x: Double) = java.lang.Double.valueOf(x)
implicit def boolean2Boolean(x: Boolean) = java.lang.Boolean.valueOf(x)
implicit def Byte2byte(x: java.lang.Byte): Byte = x.byteValue
implicit def Short2short(x: java.lang.Short): Short = x.shortValue
implicit def Character2char(x: java.lang.Character): Char = x.charValue
implicit def Integer2int(x: java.lang.Integer): Int = x.intValue
implicit def Long2long(x: java.lang.Long): Long = x.longValue
implicit def Float2float(x: java.lang.Float): Float = x.floatValue
implicit def Double2double(x: java.lang.Double): Double = x.doubleValue
implicit def Boolean2boolean(x: java.lang.Boolean): Boolean = x.booleanValue
Implicits are designed for such purpose. The compiler will search if any implicit definition objects are available. If it finds then will use them. There are many implicit definition objects provided scala. You can also define the custom objects to be implicit and will be used if they are within the scope of your code.
See the doc
https://docs.scala-lang.org/tour/implicit-parameters.html
doer works because since it accepts implicit parameter, it is available in Predef. Predef is imported automatically to scala scope. Looks like the function used is int2Integer.
I'm trying to write a function which re-uses the implicit conversions which I have for Object A -> Object B when they are wrapped in an Option in a generic way so that Option[A] -> Option[B] conversions also work.
What I've come up with is:
implicit def fromOptionToOption[A, B](from: Option[A])(implicit conversion: (A) => B): Option[B] = from.map(conversion(_))
This works when I assign a Some(..) to a value but not when I assign an Option val; see the following console output:
scala> trait T
defined trait T
scala> case class Foo(i: Int) extends T
defined class Foo
scala> case class Bar(i: Int) extends T
defined class Bar
scala> implicit def fromFooToBar(f: Foo):Bar = Bar(f.i)
fromFooToBar: (f: Foo)Bar
scala> implicit def fromBarToFoo(b: Bar):Foo = Foo(b.i)
fromBarToFoo: (b: Bar)Foo
scala> implicit def fromOptionToOption[A, B](from: Option[A])(implicit conversion: (A) => B): Option[B] = from.map(conversion(_))
fromOptionToOption: [A, B](from: Option[A])(implicit conversion: (A) => B)Option[B]
scala> val foo: Option[Foo] = Some(Bar(1))
foo: Option[Foo] = Some(Foo(1))
// THIS WORKS as expected
scala> val fooOpt = Some(Foo(4))
fooOpt: Some[Foo] = Some(Foo(4))
scala> val barOpt2: Option[Bar] = fooOpt
<console>:16: error: type mismatch;
found : Some[Foo]
required: Option[Bar]
val barOpt2: Option[Bar] = fooOpt
^
//THIS FAILS.
I don't really see the difference between the first and second conversion. Somehow it doesn't invoke the implicit conversion in the latter. I guess it has something to do with the type system, but I can't see how just yet. Any ideas?
-Albert
(I'm on scala 2.9.1)
Here's clue:
scala> val fooOpt: Option[Bar] = Option(Foo(1))
fooOpt: Option[Bar] = Some(Bar(1))
And another:
scala> implicit def foobar(x: String): Int = augmentString(x).toInt
foobar: (x: String)Int
scala> val y: Option[String] = Option(1)
y: Option[String] = Some(1)
scala> val y: Option[Int] = Option("1")
y: Option[Int] = Some(1)
Looks like a legitimately odd bug. I'd pop open a smaller test case and open an issue (or search for one in JIRA).
As an aside:
You could use some category theory to handle lots of different types of "Option-ish" things.
package object fun {
trait Functor[Container[_]] {
def fmap[A,B](x: Container[A], f: A => B): Container[B]
}
object Functor {
implicit object optionFunctor extends Functor[Option] {
override def fmap[A,B](x: Option[A], f: A => B): Option[B] = x map f
}
// Note: With some CanBuildFrom magic, we can support Traversables here.
}
implicit def liftConversion[F[_], A, B](x: F[A])(implicit f: A => B, functor: Functor[F]): F[B] =
functor.fmap(x,f)
}
That's a bit more advanced, as you're mapping some category theory FP onto the problem, but it's a more general solution to lift implicit conversations into containers as needed. Notice how they chain by using one implicit conversation method that takes a more limited implicit argument.
ALSO, this should make the examples work:
scala> val tmp = Option(Foo(1))
tmp: Option[Foo] = Some(Foo(1))
scala> val y: Option[Bar] = tmp
y: Option[Bar] = Some(Bar(1))
And make your usage of Some more dangerous:
scala> val tmp = Some(Foo(1))
tmp: Some[Foo] = Some(Foo(1))
scala> val y: Option[Bar] = tmp
<console>:25: error: could not find implicit value for parameter functor: fun.Functor[Some]
val y: Option[Bar] = tmp
^
That's telling you that variance is critical, and interacts with implicits. My guess is you ran into a very rare, probably hard to fix bug that can be avoided using other techniques.
You might not be aware of it, but there's a flag for that: -Xlog-implicits. And this is what it says:
scala> val barOpt2: Option[Bar] = fooOpt
fromOptionToOption is not a valid implicit value for Some[Foo] => Option[Bar] because:
incompatible: (from: Option[Foo])(implicit conversion: Foo => B)Option[B] does not match expected type Some[Foo] => Option[Bar]
<console>:16: error: type mismatch;
found : Some[Foo]
required: Option[Bar]
val barOpt2: Option[Bar] = fooOpt
^
And there you go -- it doesn't know what type B must be. 0__ mentioned that this problem doesn't happen with invariant collections, and that makes some sense. In invariant collections, B must be exactly Bar, while for covariant collections it could be any subtype of Bar.
So, why does val foo: Option[Foo] = Some(Bar(1)) work? Well, there's a flag for that too... -Ytyper-debug. Not for the weak, however, given the extreme verbosity.
I waddled through anyway, comparing what happens in both cases, and the answer is rather simple... it's not the Option that is being converted in that case, but Bar! Remember, you declared an implicit conversion from Bar => Foo, so it is applying that conversion before passing the result to Some!
It doesn't work because the Scala Language Specification defines view as follows:
Implicit parameters and methods can also define implicit conversions called views. A view from type S to type T is defined by an implicit value which has function type S=>T or (=>S)=>T or by a method convertible to a value of that type.
fromOptionToOption doesn't conform to the three categories since it takes an implicit parameter. Compiler doesn't seem to find converter with both destination and source having generic type.
Defining a view from Option[Foo] to Option[Bar] works as expected.
trait T
case class Foo(i: Int) extends T
case class Bar(i: Int) extends T
object Main {
implicit def fromFooToBar(f: Foo):Bar = Bar(f.i)
implicit def fromBarToFoo(b: Bar):Foo = Foo(b.i)
// implicit def fromOptionToOption[A, B](from: Option[A])(implicit conversion: (A) => B): Option[B] =
// from.map(conversion(_))
implicit def fromOptionFooToOptionBar(o: Option[Foo]): Option[Bar] = o map { foo => foo }
def test(): Option[Bar] = {
val fooOpt = Some(Foo(4))
val barOpt2: Option[Bar] = fooOpt
barOpt2
}
}
println(Main.test)
Running this prints out:
$ scala so.scala
Some(Bar(4))
However, all is not lost. It's not as nice as general Option to Option, but we can do something like anything that can turn into Bar to Option[Bar] by view bound.
trait T
case class Foo(i: Int) extends T
case class Bar(i: Int) extends T
object Main {
implicit def fromFooToBar(f: Foo):Bar = Bar(f.i)
implicit def fromBarToFoo(b: Bar):Foo = Foo(b.i)
implicit def fromOptionToOptionBar[A <% Bar](from: Option[A]): Option[Bar] =
from map { foo => foo }
def test(): Option[Bar] = {
val fooOpt = Some(Foo(4))
val barOpt2: Option[Bar] = fooOpt
barOpt2
}
}
println(Main.test)
Here's another workaround that can be used for general Option to Option but requires extra .convert call:
trait T
case class Foo(i: Int) extends T
case class Bar(i: Int) extends T
case class Converter[A](x: Option[A]) {
def convert[B](implicit ev: Function1[A, B]): Option[B] = x map { a: A => ev(a) }
}
object Main {
implicit def optionToConverter[A](x: Option[A]) = Converter(x)
implicit def fooToBar(x: Foo) = Bar(x.i)
def test(): Option[Bar] = {
val fooOpt = Some(Foo(4))
val barOpt: Option[Bar] = fooOpt.convert
barOpt
}
}
println(Main.test)
Indeed it's a very strange problem. I tried to use another type than Option, and it turns out that the problem is that Option is covariant in its type parameter. This works all:
case class A[B](value: B) // invariant in B
case class X()
case class Y()
implicit def xtoy(x: X): Y = Y()
implicit def ytox(x: Y): X = X()
implicit def movea[U, V](from: A[U])(implicit view: U => V): A[V] = A[V](from.value)
def test(a: A[Y]) = "ok"
test(A(X())) // (1)
val f = A(X())
test(f) // (2)
But if instead I define A as
case class A[+B](value: B) // covariant in B
The case (2) fails. Case (1) always succeeds, because Scala already converts X to Y before wrapping it in an A.
Now that we know the problem source, you need to wait for a type guru to explain why this is actually a problem... The conversion is still valid, you see:
askForY(movea(f)) // succeeds, even with A[+B]
I improved #jseureth answer and added support for Traversable:
trait Mappable[A, B, C[_]] {
def apply(f: A => B): C[B]
}
package object app {
implicit class OptionMappable[A, B, C[X] <: Option[X]](option: C[A]) extends Mappable[A, B, Option] {
override def apply(f: A => B): Option[B] = option.map(f)
}
implicit class TraversableMappable[A, B, C[X] <: Traversable[X]](traversable: C[A])
(implicit cbf: CanBuildFrom[C[A], B, C[B]]) extends Mappable[A, B, C] {
override def apply(f: A => B): C[B] = {
val builder = cbf(traversable)
builder.sizeHint(traversable)
builder ++= traversable.map(f)
builder.result()
}
}
implicit def liftConversion[C[_], A, B](x: C[A])
(implicit f: A => B, m: C[A] => Mappable[A, B, C]): C[B] = m(x)(f)
}
Now you can implicitly convert options and traversables:
implicit def f(i: Int): String = s"$i"
val a: Option[String] = Some(1)
val b: Seq[String] = Seq(1, 2, 3)