Static imports inside scala REPL - scala

Given the following object:
scala> object P2pClient {
| type Num = Double
| type Weights = Array[Array[Num]]
| }
defined object P2pClient
and the following import:
import P2pClient._
The Weights type seems to be properly understood:
val w: Weights = new Weights(3)
w: P2pClient.Weights = Array(null, null, null)
But then why is it not working in the following construct:
case class SendWeightsReq(W: Weights) extends P2pReq[Weights] {
| override def value() = W
| }
<console>:12: error: not found: type Weights
case class SendWeightsReq(W: Weights) extends P2pReq[Weights] {
^
<console>:12: error: not found: type Weights
case class SendWeightsReq(W: Weights) extends P2pReq[Weights] {
^
Any ideas on what were happening here (/workaround) ?
Update There appear to be significant limitations on wildcard imports int he REPL. Here is another simpler illustration:
scala> import reflect.runtime.universe._
import reflect.runtime.universe._
scala> trait TT { def x[T <: java.io.Serializable : TypeTag]: T }
<console>:10: error: not found: type TypeTag
trait TT { def x[T <: java.io.Serializable : TypeTag]: T }
^
So we see that the wildcard import did not work. Here is the same code with the explicit package:
scala> trait TT { def x[T <: java.io.Serializable : reflect.runtime.universe.TypeTag]: T }
defined trait TT

I see the issue is due to my using the Spark REPL spark-shell. The issue is not happening in the normal Scala REPL.

Good news: your code works! Bad news: I have no idea why it doesn't work for you.
Here is my REPL session; I was going to build up to your example and see what broke, but nothing did.
scala> object P2pClient { type Num = Int; type Weights = Array[Array[Num]] }
defined object P2pClient
scala> import P2pClient._
import P2pClient._
scala> val x = new Weights(3)
x: Array[Array[P2pClient.Num]] = Array(null, null, null)
scala> case class SendWeights(W: Weights)
defined class SendWeights
scala> val s = new SendWeights(new Weights(3))
s: SendWeights = SendWeights([[I#1cab0bfb)
Hmmm. Trivial example works.
scala> case class SendWeights2(w: Weights) { def dubs = w }
defined class SendWeights2
Works when it has a body.
scala> trait P2pReq[+T] { def value(): Unit }
defined trait P2pReq
scala> case class SendWeights3(W: Weights) extends P2pReq[Weights] {
override def value() = W }
defined class SendWeights3
scala> val foo = new SendWeights3(new Weights(3))
foo: SendWeights3 = SendWeights3([[I#51dcb805)
res2: P2pClient.Weights = Array(null, null, null)
Aaaaand works when it extends a polymorphic trait and overrides one of its members. The only thing that may be different is your definition of P2pReq, but I don't see how that could cause one of your types to go unrecognized by the interpreter.

Related

Compile error with Tagged Type and Play Json Format typeclass derivation

In the following example, I want to use tagged types for the ids of my classes. I've created an utility trait to reduce some boilerplate (of tags/reads/writes declaration):
import java.util.UUID
import play.api.libs.json.{Format, Json, Reads, Writes}
trait Opaque[A] {
protected type Tagged[U] = { type Tag = U }
type ##[U, T] = U with Tagged[T]
trait Tag
def tag(a: A): A ## Tag = a.asInstanceOf[A ## Tag]
def untag(a: A ## Tag): A = a
implicit def reads(implicit r: Reads[A]): Reads[A ## Tag] =
r.map(tag)
implicit def writes(implicit w: Writes[A]): Writes[A ## Tag] =
w.contramap(untag)
implicit def format(implicit r: Reads[A], w: Writes[A]): Format[A ## Tag] =
Format(reads(r), writes(w))
}
final case class Foo(id: Foo.FooId.T, f1: Boolean)
object Foo {
object FooId extends Opaque[UUID] {
type T = UUID ## Tag
}
import FooId._
implicit val fmt: Format[Foo] = Json.format[Foo]
}
final case class Bar(id: Bar.BarId.T, fooId: Foo.FooId.T, b1: String)
object Bar {
object BarId extends Opaque[UUID] {
type T = UUID ## Tag
}
import Foo.FooId._
import BarId._
implicit val format: Format[Bar] = Json.format[Bar]
}
I have the following error from the compiler:
implicit val format: Format[Bar] = Json.format[Bar]
^
<pastie>:43: error: No instance of play.api.libs.json.Format is available for Opaque.<refinement>, Opaque.<refinement> in the implicit scope (Hint: if declared in the same file, make sure it's declared before)
I'm not able to explain why I'm having this behaviour, the error message is not explicit. I'm importing the Format for FooId and BarId needed for deriving a format for Bar class.
The thing is that names of implicits are significant.
Very simple example of that is following:
object MyObject {
implicit val i: Int = ???
}
import MyObject._
implicit val i: String = ???
// implicitly[Int] // doesn't compile
// implicitly[String] // doesn't compile
but
object MyObject {
implicit val i: Int = ???
}
import MyObject._
implicit val i1: String = ???
implicitly[Int] // compiles
implicitly[String] // compiles
If you want derivation Json.format[Bar] to work, there should be implicits Format[Bar.BarId.T], Format[Foo.FooId.T] in scope i.e. Format instances for fields of Bar. If you make the only import
import Foo.FooId._
implicitly[Format[Foo.FooId.T]] // compiles
and
import BarId._
implicitly[Format[Bar.BarId.T]] // compiles
but if you import both, since names of implicits collide
import Foo.FooId._
import BarId._
// implicitly[Format[Foo.FooId.T]] // doesn't compiles
// implicitly[Format[Bar.BarId.T]] // doesn't compiles
For example you can move trait Tag outside trait Opaque and make the only import. Then
implicitly[Format[Foo.FooId.T]]
implicitly[Format[Bar.BarId.T]]
Json.format[Bar]
will compile.
https://youtu.be/1h8xNBykZqM?t=681 Some Mistakes We Made When Designing Implicits, Mistake #1
NullPointerException on implicit resolution

Scala: How can an import prevent finding an implicit value?

I could use suggestions debugging an implicit:
I want to use the implicit, x:
type T
trait HasT {
implicit def x: T = ...
}
But I also need a wildcard import from some package foo. I've tried two different ways of introducing both:
class UseT extends HasT {
import foo._
implicitly[T] // fails! "could not find implicit value"
// use foo stuff
}
and
class UseT {
object hasT extends HasT
import hasT.x
import foo._
implicitly[T] // fails! "could not find implicit value"
}
Both fail with "could not find" (not "ambiguous implicits values").
This happens while implicit identifier x: T is accessible at the point of method call via either inheritance or importing.
My workaround is to rebind x to an implicit val before the import. Both of the following work:
implicit val x2: T = implicitly[T]
import foo._
implicitly[T] // works!
and
implicit val x2: T = x
import foo._
implicitly[T] // works!
What value could possibly be in foo to cause this behavior?
My first guess is that there is some competing implicit in foo, but if it were higher priority, the following implicitly would still work, and if it were an ambiguous implicit, I'd be getting a different a different error.
edit: Miles Sabin's guess was correct! I found the shadowing implicit: timeColumnType. I still don't completely understand how this works given Som Snytt's observation that the shadowing implicit was wildcard imported (lower priority) while the shadowed was inherited (part of highest priority), so I'll leave the whole post here for posterity.
retracted: A second guess, offered by miles sabin, is implicit shadowing. I've since clarified my post to exclude that possibility. That case would be consistent with my errors if I had tried package hasT extends HasT; import hasT._, but As som-snytt points out, those two cases would not result in shadowing.
In my specific case, this can be confirmed by changing the name of the implicit I'm trying to use.
(This is false. I likely missed a publishLocal or reload while using this test to verify.)
context: I'm actually trying to use slick. The implicit T above is actually a column type mapping:
import slick.driver.JdbcProfile
class Custom { ... } // stored as `Long` in postgres
trait ColumnTypes {
val profile: JdbcProfile
import profile.api._ // this is `foo` above
type T = profile.BaseColumnType[Custom]
implicit def customColumnType: T =
MappedColumnType.base[Custom, Long](_.toLong, Custom.fromLong)
}
class DatabaseSchema(val profile: JdbcProfile) extends ColumnTypes {
// `implicitly[T]` does not fail here.
import profile.api._ // this is also `foo` above
// `implicitly[T]` fails here, but it's needed for the following:
class CustomTable(tag: Tag) extends Table[Custom](tag, "CUSTOMS") {
// following fails unless I rebind customColumnType to a local implicit
def custom = column[Custom]("CUSTOM")
def * = custom
}
}
The type of api/foo is JdbcProfile.API. The offending implicit is probably here, but I can't tell why. I'll try blocking some of those from being imported and see if I can narrow it down.
I think it's most likely that foo contains a definition named x. When (wildcard) imported from foo it shadows the local definition,
scala> object foo { val x: Boolean = true }
defined object foo
scala> implicit val x: Int = 23
x: Int = 23
scala> implicitly[Int]
res0: Int = 23
scala> import foo._
import foo._
scala> implicitly[Int]
<console>:17: error: could not find implicit value for parameter e: Int
implicitly[Int]
^
This is clearly a bug in implicit search.
First, eligible are all identifiers x that can be accessed at the
point of the method call without a prefix and that denote an implicit
definition or an implicit parameter. An eligible identifier may thus
be a local name, or a member of an enclosing template, or it may be
have been made accessible without a prefix through an import clause.
In the example, unprefixed x refers to the inherited symbol. X.x is not accessible without a prefix.
Implicit search is fumbling the import.
$ scala
Welcome to Scala 2.12.0 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_111).
Type in expressions for evaluation. Or try :help.
scala> :pa
// Entering paste mode (ctrl-D to finish)
object X { def x: Int = 42 }
trait T { def x: Int = 17 }
object Y extends T {
import X._
def f = x
}
// Exiting paste mode, now interpreting.
defined object X
defined trait T
defined object Y
scala> Y.f
res0: Int = 17
scala> :pa
// Entering paste mode (ctrl-D to finish)
object X { implicit def x: Int = 42 }
trait T { implicit def x: Int = 17 }
object Y extends T {
import X._
def f: Int = implicitly[Int]
}
// Exiting paste mode, now interpreting.
<pastie>:19: error: could not find implicit value for parameter e: Int
def f: Int = implicitly[Int]
^
scala> :pa
// Entering paste mode (ctrl-D to finish)
object X { implicit def x: Int = 42 }
trait T { implicit def x: Int = 17 }
object Y extends T {
import X.{x => _, _} // avoids bug, but is redundant
def f: Int = implicitly[Int]
}
// Exiting paste mode, now interpreting.
defined object X
defined trait T
defined object Y
scala>
The other example using the REPL is scoped this way:
scala> :pa
// Entering paste mode (ctrl-D to finish)
object X { def x: Int = 42 }
object Y { implicit def x: Int = 17 }
object Z {
import Y.x
def f = {
import X._
x
}
}
// Exiting paste mode, now interpreting.
<pastie>:19: error: reference to x is ambiguous;
it is imported twice in the same scope by
import X._
and import Y.x
x
^
Where x is not available at all, and the implicit is correctly excluded.
Just to confirm:
$ scala -Xlog-implicits
Welcome to Scala 2.12.0 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_111).
Type in expressions for evaluation. Or try :help.
scala> :pa
// Entering paste mode (ctrl-D to finish)
object X { implicit def x: Int = 42 }
trait T { implicit def x: Int = 17 }
object Y extends T {
import X._
def f: Int = implicitly[Int]
}
// Exiting paste mode, now interpreting.
<console>:17: x is not a valid implicit value for Int because:
candidate implicit method x in object X is shadowed by method x in trait T
def f: Int = implicitly[Int]
^
<console>:17: x is not a valid implicit value for Int because:
candidate implicit method x in object X is shadowed by method x in trait T
def f: Int = implicitly[Int]
^
<console>:17: error: could not find implicit value for parameter e: Int
def f: Int = implicitly[Int]
^
scala>
Probably https://issues.scala-lang.org/browse/SI-9208

enable implicit import for runtime type in scala check. "could not find implicit value for parameter"

I'm testing my own home-brewed Monoid classes in scala using the ScalaCheck library and ScalaTest
when attempting to implement DRY tests, I get the implicit error in the title:
Error:(16, 12) could not find implicit value for parameter arbA: org.scalacheck.Arbitrary[A]
forAll { (a: A) =>
^
here is the implementation of intAddition Monoid:
trait Monoid[A] {
def op(a1: A, a2: A): A
def zero: A
}
object Monoid {
...
val intAddition: Monoid[Int] = new Monoid[Int] {
override def op(a1: Int, a2: Int): Int = a1 + a2
override def zero: Int = 0
}
...
}
And the test suite:
import org.fpinscala.monoids.Monoid._
import org.fpinscala.testutils.UnitSpec
import org.scalatest.prop.PropertyChecks
import org.scalacheck.Arbitrary._
import scala.language.implicitConversions
class MonoidSpec extends UnitSpec with PropertyChecks {
def assertIdentityBehaviour[A](M: Monoid[A]): Unit = {
import M._
forAll { (a: A) =>
op(zero, a) should be(a)
op(a, zero) should be(a)
}
}
behavior of "intAdditionMonoid"
it should "obey identity laws" in {
assertIdentityBehaviour(intAddition)
}
}
This code compiles but fails at runtime (runtime type erasure?).
Is what I'm trying to achieve possible in Scala?
This code compiles
It doesn't; the error you give is a compilation error. It should be fixed by adding the implicit parameter it complains about:
def assertIdentityBehaviour[A](M: Monoid[A])(implicit arbA: Arbitrary[A]) = ...
// or equivalently, def assertIdentityBehaviour[A: Arbitrary](M: Monoid[A]) = ...
You are calling assertIdentityBehaviour only with A for which the parameter is available, but the error is in its definition.

Scala Macros: Getting a List of TypeSymbols to be used at runtime

Is there a way to return a List of TypeSymbols for each class under a package using macros?
What I am trying to achieve is to write a macro that gives out something equivalent to this list:
scala> import scala.reflect.runtime.universe._
import scala.reflect.runtime.universe._
scala> case class MyClass1()
defined class MyClass1
scala> case class MyClass2()
defined class MyClass2
scala> val typeSymbols = List(typeOf[MyClass1].typeSymbol, typeOf[MyClass2].typeSymbol)
typeSymbols: List[reflect.runtime.universe.Symbol] = List(class MyClass1, class MyClass2)
Here's my setup:
I have a package named foo, under which these are defined:
trait FooTrait
case class Bar() extends FooTrait
case class Bar() extends FooTrait
Here's my macro that gets all type symbols for the classes under foo that extend FooTrait:
def allTypeSymbols_impl[T: c.WeakTypeTag](c: Context)(packageName: c.Expr[String]) = {
import c.universe._
// Get package name from the expression tree
val Literal(Constant(name: String)) = packageName.tree
// Get all classes under given package name
val pkg = c.mirror.staticPackage(name)
// Obtain type symbols for the classes - implementation omitted
val types = getTypeSymbols(c.universe)(List(pkg))
// Apply method for List. For easy readability in later applications
val listApply = Select(reify(List).tree, newTermName("apply"))
val result = types.map {
t =>
val typeName = c.Expr[TypeSymbol](Ident(t))
println(s"Typename: $typeName, $t, ${t.toType}")
reify(typeName.splice).tree
}
println(s"RESULT: ${showRaw(result)}")
c.Expr[List[reflect.runtime.universe.TypeSymbol]](Apply(listApply, result.toList))
}
The first println prints:
Typename: Expr[c.universe.TypeSymbol](Bar), class Bar, foo.Bar
Typename: Expr[c.universe.TypeSymbol](Baz), class Baz, foo.Baz
The second one prints:
RESULT: List(Ident(foo.Bar), Ident(foo.Baz))
But I get this error message:
[error] no type parameters for method any2ArrowAssoc: (x: A)ArrowAssoc[A] exist so that it can be applied to arguments (<notype>)
[error] --- because ---
[error] argument expression's type is not compatible with formal parameter type;
[error] found : <notype>
[error] required: ?A
[error] Note that <none> extends Any, not AnyRef.
[error] Such types can participate in value classes, but instances
[error] cannot appear in singleton types or in reference comparisons.
What should I do to make this work? I suspect that I have to write something else instead of Ident, but I couldn't figure out what.
Using Scala 2.10.2.
Thanks in advance!
You have to use reifyType to create reflection artifacts in the runtime universe:
import scala.language.experimental.macros
import scala.reflect.macros.Context
object PackageMacros {
def allTypeSymbols[T](packageName: String) = macro allTypeSymbols_impl[T]
def allTypeSymbols_impl[T: c.WeakTypeTag](c: Context)(
packageName: c.Expr[String]
) = {
import c.universe._
val pkg = packageName.tree match {
case Literal(Constant(name: String)) => c.mirror.staticPackage(name)
}
val types = pkg.typeSignature.members.collect {
case sym: ClassSymbol =>
c.reifyType(treeBuild.mkRuntimeUniverseRef, EmptyTree, sym.toType)
}.toList
val listApply = Select(reify(List).tree, newTermName("apply"))
c.Expr[List[Any]](Apply(listApply, types))
}
}
This will give you a list of type tags, not symbols, but you can pretty easily get the symbols, either like this:
scala> PackageMacros.allTypeSymbols("foo").map(_.tpe.typeSymbol) foreach println
class Baz$
class Bar
class Baz
trait FooTrait
class Bar$
Or in the macro itself.

Path-Dependent type inside class value in Scala

I would like to give a value of a type with an abstract type to a class and later use it's path dependent type. Look at the following example (using Scala 2.10.1):
trait Foo {
type A
def makeA: A
def useA(a: A): Unit
}
object Test {
class IntFoo extends Foo {
type A = Int
def makeA = 1
def useA(a: Int) = println(a)
}
class FooWrap(val a: Foo) {
def wrapUse(v: a.A) = a.useA(v)
}
val foo = new IntFoo
/* Path dependent locally */
val bar = foo
bar.useA(foo.makeA) // works
/* Path dependent through class value */
val fooWrap = new FooWrap(foo)
fooWrap.a.useA(foo.makeA) // fails
// error: type mismatch; found : Int required: Test.fooWrap.a.A
fooWrap.wrapUse(foo.makeA) // fails
// error: type mismatch; found : Int required: Test.fooWrap.a.A
}
First, I do not understand the fundamental difference between the local and the class-value case (note the public, immutable value) and why the type checking fails (because obviously Test.fooWrap.a.A =:= foo.A). Is this a limitation of the Scala compiler?
Second, how can I achieve what I am trying to do?
UPDATE
It seems that this can be achieved by using generics and inline type-constraints:
class FooWrap[T](val a: Foo { type A = T }) {
def wrapUse(v: T) = a.useA(v)
}
However, in my case, A is actually a higher-kinded type, so the example becomes:
trait Foo {
type A[T]
def makeA[T]: A[T]
def useA(a: A[_]): Unit
}
object Test {
class OptFoo extends Foo {
type A[T] = Option[T]
def makeA[T] = None
def useA(a: A[_]) = println(a.get)
}
class FooWrap(val a: Foo) {
def wrapUse(v: a.A[_]) = a.useA(v)
}
val foo = new OptFoo
/* Path dependent locally (snip) */
/* Path dependent through class value */
val fooWrap = new FooWrap(foo)
fooWrap.a.useA(foo.makeA) // fails
// polymorphic expression cannot be instantiated to expected type;
// found : [T]None.type required: Test.fooWrap.a.A[_]
fooWrap.wrapUse(foo.makeA) // fails
// polymorphic expression cannot be instantiated to expected type;
// found : [T]None.type required: Test.fooWrap.a.A[_]
}
In your original question, your problem is that the Scala compiler is unable to prove equality of the result type of foo.makeA with the argument type of fooWrap.a.useA. To do that it would need to be able to prove the identity of foo with fooWrap.a which we can intuitively see must be the case here, but which isn't straightforward for the compiler to track.
There are a couple of ways to work around this problem. First, you could use fooWrap.a uniformly in place of foo,
scala> fooWrap.a.useA(fooWrap.a.makeA)
1
Now it's simple for the compiler to recognize the prefix of A (fooWrap.a) as being the same in both occurrences.
Second, you could parametrize FooWrap in a way which captures the type of its Foo argument more precisely,
scala> class FooWrap[F <: Foo](val a: F) {
| def wrapUse(v: a.A) = a.useA(v)
| }
defined class FooWrap
scala> val fooWrap = new FooWrap(foo)
fooWrap: FooWrap[IntFoo] = FooWrap#6d935671
scala> fooWrap.a.useA(foo.makeA)
1
Here the type argument of FooWrap is inferred as IntFoo rather than as bare Foo, hence A is known to be exactly Int, as it is in the result type of foo.makeA.
In your update you introduce an additional wrinkle: you change the signature of useA to,
def useA(a: A[_]): Unit
The _ here is an existential which will frustrate all attempts to coax the compiler into proving useful type equalities. Instead you need something along the lines of,
trait Foo {
type A[T]
def makeA[T]: A[T]
def useA[T](a: A[T]): Unit
}
class OptFoo extends Foo {
type A[T] = Option[T]
def makeA[T]: A[T] = None
def useA[T](a: A[T]) = a map println
}
class FooWrap[F <: Foo](val a: F) {
def wrapUse[T](v: a.A[T]) = a.useA(v)
}
val foo = new OptFoo
Sample REPL session,
scala> val fooWrap = new FooWrap(foo)
fooWrap: FooWrap[OptFoo] = FooWrap#fcc10a7
scala> fooWrap.a.useA(foo.makeA)
scala>
The higher kinded-type can be added to FooWrap as generic parameter, too:
class FooWrap[T[V]](val a: Foo { type A[V] = T[V] }) {
def wrapUse(v: T[_]) = a.useA(v)
}
but (in this example) the inference fails:
val fooWrap = new FooWrap[Option](foo)
Otherwise:
- type mismatch; found : Test.foo.type (with underlying type Test.OptFoo) required: Foo{type A[V] = T[V]}
- inferred kinds of the type arguments (Option[V]) do not conform to the expected kinds of the type parameters (type T) in class FooWrap. Option[V]'s type parameters do not match type T's expected
parameters: class Option has one type parameter, but type T has one
Any other, nicer solutions?