Scala: How can an import prevent finding an implicit value? - scala

I could use suggestions debugging an implicit:
I want to use the implicit, x:
type T
trait HasT {
implicit def x: T = ...
}
But I also need a wildcard import from some package foo. I've tried two different ways of introducing both:
class UseT extends HasT {
import foo._
implicitly[T] // fails! "could not find implicit value"
// use foo stuff
}
and
class UseT {
object hasT extends HasT
import hasT.x
import foo._
implicitly[T] // fails! "could not find implicit value"
}
Both fail with "could not find" (not "ambiguous implicits values").
This happens while implicit identifier x: T is accessible at the point of method call via either inheritance or importing.
My workaround is to rebind x to an implicit val before the import. Both of the following work:
implicit val x2: T = implicitly[T]
import foo._
implicitly[T] // works!
and
implicit val x2: T = x
import foo._
implicitly[T] // works!
What value could possibly be in foo to cause this behavior?
My first guess is that there is some competing implicit in foo, but if it were higher priority, the following implicitly would still work, and if it were an ambiguous implicit, I'd be getting a different a different error.
edit: Miles Sabin's guess was correct! I found the shadowing implicit: timeColumnType. I still don't completely understand how this works given Som Snytt's observation that the shadowing implicit was wildcard imported (lower priority) while the shadowed was inherited (part of highest priority), so I'll leave the whole post here for posterity.
retracted: A second guess, offered by miles sabin, is implicit shadowing. I've since clarified my post to exclude that possibility. That case would be consistent with my errors if I had tried package hasT extends HasT; import hasT._, but As som-snytt points out, those two cases would not result in shadowing.
In my specific case, this can be confirmed by changing the name of the implicit I'm trying to use.
(This is false. I likely missed a publishLocal or reload while using this test to verify.)
context: I'm actually trying to use slick. The implicit T above is actually a column type mapping:
import slick.driver.JdbcProfile
class Custom { ... } // stored as `Long` in postgres
trait ColumnTypes {
val profile: JdbcProfile
import profile.api._ // this is `foo` above
type T = profile.BaseColumnType[Custom]
implicit def customColumnType: T =
MappedColumnType.base[Custom, Long](_.toLong, Custom.fromLong)
}
class DatabaseSchema(val profile: JdbcProfile) extends ColumnTypes {
// `implicitly[T]` does not fail here.
import profile.api._ // this is also `foo` above
// `implicitly[T]` fails here, but it's needed for the following:
class CustomTable(tag: Tag) extends Table[Custom](tag, "CUSTOMS") {
// following fails unless I rebind customColumnType to a local implicit
def custom = column[Custom]("CUSTOM")
def * = custom
}
}
The type of api/foo is JdbcProfile.API. The offending implicit is probably here, but I can't tell why. I'll try blocking some of those from being imported and see if I can narrow it down.

I think it's most likely that foo contains a definition named x. When (wildcard) imported from foo it shadows the local definition,
scala> object foo { val x: Boolean = true }
defined object foo
scala> implicit val x: Int = 23
x: Int = 23
scala> implicitly[Int]
res0: Int = 23
scala> import foo._
import foo._
scala> implicitly[Int]
<console>:17: error: could not find implicit value for parameter e: Int
implicitly[Int]
^

This is clearly a bug in implicit search.
First, eligible are all identifiers x that can be accessed at the
point of the method call without a prefix and that denote an implicit
definition or an implicit parameter. An eligible identifier may thus
be a local name, or a member of an enclosing template, or it may be
have been made accessible without a prefix through an import clause.
In the example, unprefixed x refers to the inherited symbol. X.x is not accessible without a prefix.
Implicit search is fumbling the import.
$ scala
Welcome to Scala 2.12.0 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_111).
Type in expressions for evaluation. Or try :help.
scala> :pa
// Entering paste mode (ctrl-D to finish)
object X { def x: Int = 42 }
trait T { def x: Int = 17 }
object Y extends T {
import X._
def f = x
}
// Exiting paste mode, now interpreting.
defined object X
defined trait T
defined object Y
scala> Y.f
res0: Int = 17
scala> :pa
// Entering paste mode (ctrl-D to finish)
object X { implicit def x: Int = 42 }
trait T { implicit def x: Int = 17 }
object Y extends T {
import X._
def f: Int = implicitly[Int]
}
// Exiting paste mode, now interpreting.
<pastie>:19: error: could not find implicit value for parameter e: Int
def f: Int = implicitly[Int]
^
scala> :pa
// Entering paste mode (ctrl-D to finish)
object X { implicit def x: Int = 42 }
trait T { implicit def x: Int = 17 }
object Y extends T {
import X.{x => _, _} // avoids bug, but is redundant
def f: Int = implicitly[Int]
}
// Exiting paste mode, now interpreting.
defined object X
defined trait T
defined object Y
scala>
The other example using the REPL is scoped this way:
scala> :pa
// Entering paste mode (ctrl-D to finish)
object X { def x: Int = 42 }
object Y { implicit def x: Int = 17 }
object Z {
import Y.x
def f = {
import X._
x
}
}
// Exiting paste mode, now interpreting.
<pastie>:19: error: reference to x is ambiguous;
it is imported twice in the same scope by
import X._
and import Y.x
x
^
Where x is not available at all, and the implicit is correctly excluded.
Just to confirm:
$ scala -Xlog-implicits
Welcome to Scala 2.12.0 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_111).
Type in expressions for evaluation. Or try :help.
scala> :pa
// Entering paste mode (ctrl-D to finish)
object X { implicit def x: Int = 42 }
trait T { implicit def x: Int = 17 }
object Y extends T {
import X._
def f: Int = implicitly[Int]
}
// Exiting paste mode, now interpreting.
<console>:17: x is not a valid implicit value for Int because:
candidate implicit method x in object X is shadowed by method x in trait T
def f: Int = implicitly[Int]
^
<console>:17: x is not a valid implicit value for Int because:
candidate implicit method x in object X is shadowed by method x in trait T
def f: Int = implicitly[Int]
^
<console>:17: error: could not find implicit value for parameter e: Int
def f: Int = implicitly[Int]
^
scala>
Probably https://issues.scala-lang.org/browse/SI-9208

Related

Compile error with Tagged Type and Play Json Format typeclass derivation

In the following example, I want to use tagged types for the ids of my classes. I've created an utility trait to reduce some boilerplate (of tags/reads/writes declaration):
import java.util.UUID
import play.api.libs.json.{Format, Json, Reads, Writes}
trait Opaque[A] {
protected type Tagged[U] = { type Tag = U }
type ##[U, T] = U with Tagged[T]
trait Tag
def tag(a: A): A ## Tag = a.asInstanceOf[A ## Tag]
def untag(a: A ## Tag): A = a
implicit def reads(implicit r: Reads[A]): Reads[A ## Tag] =
r.map(tag)
implicit def writes(implicit w: Writes[A]): Writes[A ## Tag] =
w.contramap(untag)
implicit def format(implicit r: Reads[A], w: Writes[A]): Format[A ## Tag] =
Format(reads(r), writes(w))
}
final case class Foo(id: Foo.FooId.T, f1: Boolean)
object Foo {
object FooId extends Opaque[UUID] {
type T = UUID ## Tag
}
import FooId._
implicit val fmt: Format[Foo] = Json.format[Foo]
}
final case class Bar(id: Bar.BarId.T, fooId: Foo.FooId.T, b1: String)
object Bar {
object BarId extends Opaque[UUID] {
type T = UUID ## Tag
}
import Foo.FooId._
import BarId._
implicit val format: Format[Bar] = Json.format[Bar]
}
I have the following error from the compiler:
implicit val format: Format[Bar] = Json.format[Bar]
^
<pastie>:43: error: No instance of play.api.libs.json.Format is available for Opaque.<refinement>, Opaque.<refinement> in the implicit scope (Hint: if declared in the same file, make sure it's declared before)
I'm not able to explain why I'm having this behaviour, the error message is not explicit. I'm importing the Format for FooId and BarId needed for deriving a format for Bar class.
The thing is that names of implicits are significant.
Very simple example of that is following:
object MyObject {
implicit val i: Int = ???
}
import MyObject._
implicit val i: String = ???
// implicitly[Int] // doesn't compile
// implicitly[String] // doesn't compile
but
object MyObject {
implicit val i: Int = ???
}
import MyObject._
implicit val i1: String = ???
implicitly[Int] // compiles
implicitly[String] // compiles
If you want derivation Json.format[Bar] to work, there should be implicits Format[Bar.BarId.T], Format[Foo.FooId.T] in scope i.e. Format instances for fields of Bar. If you make the only import
import Foo.FooId._
implicitly[Format[Foo.FooId.T]] // compiles
and
import BarId._
implicitly[Format[Bar.BarId.T]] // compiles
but if you import both, since names of implicits collide
import Foo.FooId._
import BarId._
// implicitly[Format[Foo.FooId.T]] // doesn't compiles
// implicitly[Format[Bar.BarId.T]] // doesn't compiles
For example you can move trait Tag outside trait Opaque and make the only import. Then
implicitly[Format[Foo.FooId.T]]
implicitly[Format[Bar.BarId.T]]
Json.format[Bar]
will compile.
https://youtu.be/1h8xNBykZqM?t=681 Some Mistakes We Made When Designing Implicits, Mistake #1
NullPointerException on implicit resolution

Pimp my library on Joda LocalDate does not work

Here I am adding a new method to to LocalDate:
object EnhancedDate {
implicit class EnhancedDate(start: org.joda.time.LocalDate) {
/** Generates dates between start and end - inclusive */
def to(end: org.joda.time.LocalDate): IndexedSeq[org.joda.time.LocalDate] = {
val numberOfDays = Days.daysBetween(start, end.plusDays(1)).getDays()
for (f <- 0 to numberOfDays) yield start.plusDays(f)
}
}
}
and now using it:
import cats.effect.IO
import EnhancedDate.EnhancedDate
def loadCadUsdRates: IO[Map[String, CadInUsd]] = IO {
val start = new org.joda.time.LocalDate(startYear,1,1)
val end = new org.joda.time.LocalDate(endYear+1,1,1).minusDays(1)
start to end map(f) toMap
}
Function f doesn't really matter; you can use identity for all it's worth bc the results are the same.
I use Idea and everything is fine in the IDE (no red underlyings) but when I run it I get this error:
Error:(104, 9) value to is not a member of org.joda.time.LocalDate
start to end map(f) toMap
Now I've used pimp my library before and I'm pretty sure this is the right way to do it, but there must be something about joda date that is incompatible with it... Have you tried pimping Joda LocalDate.. I do not know why this is not working
I'm not exactly sure what is going on here but it's something to do with the declaration being a module with the same name as the implicit class within it. This means that you have two identifiers in scope, both called EnhancedDate and I assume that this must cause resolution ambiguity. It's certainly nothing to do with Joda. One definite workaround is to place your implicit class inside the nearest package object.
package object myPackage {
implicit class EnhancedDate(start: org.joda.time.LocalDate) { ... }
}
Here's a demonstration of it not working:
scala> :paste
// Entering paste mode (ctrl-D to finish)
object Foo {
implicit class Foo(s: String) { def foo = println(s"Foo: $s") }
}
// Exiting paste mode, now interpreting.
defined object Foo
scala> import Foo._
import Foo._
scala> "hey".foo
<console>:14: error: value foo is not a member of String
"hey".foo
^
Here is it working:
scala> object Bar {
| implicit class Foo(s: String) { def foo = println(s"Foo: $s") }
| }
defined object Bar
scala> import Bar._
import Bar._
scala> "hey".foo
Foo: hey

Implicit ExecutionContext priority in Scala 2.12

In Scala 2.12 importing the global execution context and then having another implicit execution context defined in the scope results in an ambiguous implicit, while in 2.11 it works just fine.
import scala.concurrent._
import scala.concurrent.ExecutionContext.Implicits.global
class A(implicit ec: ExecutionContext) {
x = implicitly[ExecutionContext]
}
Compiler gives error:
error: ambiguous implicit values:
both lazy value global in object Implicits of type => scala.concurrent.ExecutionContext
and value ec in class A of type scala.concurrent.ExecutionContext
match expected type scala.concurrent.ExecutionContext
val x = implicitly[ExecutionContext]
^
What is the cause of this and how to work around it in code?
The spec treats overload resolution as the disambiguation of a selection of members of a class. But implicit resolution uses static overload resolution to choose between references which are not members.
Arguably, the following is a misinterpretation of the spec, since zzz is defined in a class derived from X much as yyy is:
$ scala
Welcome to Scala 2.12.0 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_101).
Type in expressions for evaluation. Or try :help.
scala> import concurrent._, ExecutionContext.global
import concurrent._
import ExecutionContext.global
scala> trait X { implicit val xxx: ExecutionContext = global }
defined trait X
scala> class Y extends X { implicit val yyy: ExecutionContext = global ; def f = implicitly[ExecutionContext] }
defined class Y
scala> class Z extends X { def f(implicit zzz: ExecutionContext) = implicitly[ExecutionContext] }
<console>:16: error: ambiguous implicit values:
both value xxx in trait X of type => scala.concurrent.ExecutionContext
and value zzz of type scala.concurrent.ExecutionContext
match expected type scala.concurrent.ExecutionContext
class Z extends X { def f(implicit zzz: ExecutionContext) = implicitly[ExecutionContext] }
^
Currently, you must rely on naming to shadow the implicit from enclosing scope:
scala> class Z extends X { def f(implicit xxx: ExecutionContext) = implicitly[ExecutionContext] }
defined class Z
Or,
scala> :pa
// Entering paste mode (ctrl-D to finish)
package object p { import concurrent._ ; implicit val xxx: ExecutionContext = ExecutionContext.global }
package p { import concurrent._ ;
class P { def f(implicit xxx: ExecutionContext) = implicitly[ExecutionContext]
def g = implicitly[ExecutionContext] }
}
// Exiting paste mode, now interpreting.
scala>

Static imports inside scala REPL

Given the following object:
scala> object P2pClient {
| type Num = Double
| type Weights = Array[Array[Num]]
| }
defined object P2pClient
and the following import:
import P2pClient._
The Weights type seems to be properly understood:
val w: Weights = new Weights(3)
w: P2pClient.Weights = Array(null, null, null)
But then why is it not working in the following construct:
case class SendWeightsReq(W: Weights) extends P2pReq[Weights] {
| override def value() = W
| }
<console>:12: error: not found: type Weights
case class SendWeightsReq(W: Weights) extends P2pReq[Weights] {
^
<console>:12: error: not found: type Weights
case class SendWeightsReq(W: Weights) extends P2pReq[Weights] {
^
Any ideas on what were happening here (/workaround) ?
Update There appear to be significant limitations on wildcard imports int he REPL. Here is another simpler illustration:
scala> import reflect.runtime.universe._
import reflect.runtime.universe._
scala> trait TT { def x[T <: java.io.Serializable : TypeTag]: T }
<console>:10: error: not found: type TypeTag
trait TT { def x[T <: java.io.Serializable : TypeTag]: T }
^
So we see that the wildcard import did not work. Here is the same code with the explicit package:
scala> trait TT { def x[T <: java.io.Serializable : reflect.runtime.universe.TypeTag]: T }
defined trait TT
I see the issue is due to my using the Spark REPL spark-shell. The issue is not happening in the normal Scala REPL.
Good news: your code works! Bad news: I have no idea why it doesn't work for you.
Here is my REPL session; I was going to build up to your example and see what broke, but nothing did.
scala> object P2pClient { type Num = Int; type Weights = Array[Array[Num]] }
defined object P2pClient
scala> import P2pClient._
import P2pClient._
scala> val x = new Weights(3)
x: Array[Array[P2pClient.Num]] = Array(null, null, null)
scala> case class SendWeights(W: Weights)
defined class SendWeights
scala> val s = new SendWeights(new Weights(3))
s: SendWeights = SendWeights([[I#1cab0bfb)
Hmmm. Trivial example works.
scala> case class SendWeights2(w: Weights) { def dubs = w }
defined class SendWeights2
Works when it has a body.
scala> trait P2pReq[+T] { def value(): Unit }
defined trait P2pReq
scala> case class SendWeights3(W: Weights) extends P2pReq[Weights] {
override def value() = W }
defined class SendWeights3
scala> val foo = new SendWeights3(new Weights(3))
foo: SendWeights3 = SendWeights3([[I#51dcb805)
res2: P2pClient.Weights = Array(null, null, null)
Aaaaand works when it extends a polymorphic trait and overrides one of its members. The only thing that may be different is your definition of P2pReq, but I don't see how that could cause one of your types to go unrecognized by the interpreter.

Test if c.universe.Type is assignable to another type in a macro

I'm attempting to write a macro which takes a class with a java bean interface and a case class and creates a pair of methods for mapping between them.
I am attempting to check that the types match for each property, however the types in the java bean are e.g. java.lang.Long and the types in the case class are scala.Long.
My question is, given the c.universe.Type object for these 2, is there a way to test if there are implicit conversions between them? i.e. to test if I can pass one to a method which expects the other.
If you want to check whether an implicit conversion exists, you can use c.inferImplicitView.
Proof of concept:
scala> :paste
// Entering paste mode (ctrl-D to finish)
import scala.reflect.macros.blackbox.Context
import scala.language.experimental.macros
def test[T,S,R](value: T, func: S => R): R = macro Macro.myMacro[T,S,R]
object Macro {
def myMacro[T: c.WeakTypeTag,S: c.WeakTypeTag,R](c: Context)(value: c.Tree, func: c.Tree): c.Tree = {
import c.universe._
val view = c.inferImplicitView(value, weakTypeOf[T], weakTypeOf[S])
if (view == EmptyTree)
c.abort(c.enclosingPosition, "Cannot apply function")
else
q"$func($value)"
}
}
// Exiting paste mode, now interpreting.
import scala.reflect.macros.blackbox.Context
import scala.language.experimental.macros
defined term macro test: [T, S, R](value: T, func: S => R)R
defined object Macro
scala> test(3L, (l: java.lang.Long) => l.toString)
res20: String = 3
scala> test(3L, (l: java.lang.Integer) => l.toString)
<console>:23: error: Cannot apply function
test(3L, (l: java.lang.Integer) => l.toString)
^
If you don't have a value, apparently it also works if you do c.inferImplicitView(EmptyTree, weakTypeOf[T], weakTypeOf[S]).
A more complex example closer to the Actual Problem:
scala> :paste
// Entering paste mode (ctrl-D to finish)
import scala.reflect.macros.blackbox.Context
import scala.language.experimental.macros
def mapBetween[JB,CC](jb: JB): CC = macro Macro.myMacro[JB,CC]
object Macro {
def myMacro[JB: c.WeakTypeTag, CC: c.WeakTypeTag](c: Context)(jb: c.Tree): c.Tree = {
import c.universe._
val jbTpe = weakTypeOf[JB]
val ccTpe = weakTypeOf[CC]
val constructor = ccTpe.members.filter(m =>
m.isConstructor && m.name != TermName("$init$")
).head.asMethod
if(constructor.paramLists.size != 1 || constructor.paramLists.head.size != 1)
c.abort(c.enclosingPosition, "not supported :(")
val ccParam = constructor.paramLists.head.head
val ccParamType = ccParam.typeSignature
val ccParamName = ccParam.name.toString
val jbGetter = jbTpe.member(TermName(s"get${ccParamName.head.toUpper + ccParamName.tail}"))
val getterType = jbGetter.asMethod.returnType
val view = c.inferImplicitView(EmptyTree, getterType, ccParamType)
if (view == EmptyTree)
c.abort(c.enclosingPosition, "Cannot apply function")
else
q"new ${ccTpe.typeSymbol.name.toTypeName}($jb.${jbGetter.name.toTermName})"
}
}
// Exiting paste mode, now interpreting.
import scala.reflect.macros.blackbox.Context
import scala.language.experimental.macros
defined term macro mapBetween: [JB, CC](jb: JB)CC
defined object Macro
scala> case class CaseClass(foo: Int)
defined class CaseClass
scala> class JavaBean{ def getFoo(): java.lang.Integer = 42 }
defined class JavaBean
scala> mapBetween[JavaBean,CaseClass](new JavaBean)
res0: CaseClass = CaseClass(42)
scala> case class CaseClass(foo: Int)
defined class CaseClass
scala> class JavaBean{ def getFoo(): java.lang.Double = 42.0 }
defined class JavaBean
scala> mapBetween[JavaBean,CaseClass](new JavaBean)
<console>:27: error: Cannot apply function
mapBetween[JavaBean,CaseClass](new JavaBean)
^