A case classes copy() method is supposed to make an identical copy of the instance, plus replacing any fields by name. This seems to fail when the case class has type parameters with manifests. The copy loses all knowledge of the types of its parameters.
case class Foo[+A : Manifest](a: A) {
// Capture manifest so we can observe it
// A demonstration with collect would work equally well
def myManifest = implicitly[Manifest[_ <: A]]
}
case class Bar[A <: Foo[Any]](foo: A) {
// A simple copy of foo
def fooCopy = foo.copy()
}
val foo = Foo(1)
val bar = Bar(foo)
println(bar.foo.myManifest) // Prints "Int"
println(bar.fooCopy.myManifest) // Prints "Any"
Why does Foo.copy lose the manifest on the parameters and how do I make it retain it?
Several Scala peculiarities interact to give this behavior. The first thing is that Manifests are not only appended to the secret implicit parameter list on the constructor but also on the copy method. It is well known that
case class Foo[+A : Manifest](a: A)
is just syntactic sugar for
case class Foo[+A](a: A)(implicit m: Manifest[A])
but this also affects the copy constructor, which would look like this
def copy[B](a: B = a)(implicit m: Manifest[B]) = Foo[B](a)(m)
All those implicit ms are created by the compiler and sent to the method through the implicit parameter list.
This would be fine as long as one was using the copy method in a place where the compiler knew Foos type parameter. For example, this will work outside of the Bar class:
val foo = Foo(1)
val aCopy = foo.copy()
println(aCopy.myManifest) // Prints "Int"
This works because the compiler infers that foo is a Foo[Int] so it knows that foo.a is an Int so it can call copy like this:
val aCopy = foo.copy()(manifest[Int]())
(Note that manifest[T]() is a function that creates a manifest representation of the type T, e.g. Manifest[T] with a capital "M". Not shown is the addition of the default parameter into copy.) It also works within the Foo class because it already has the manifest that was passed in when the class was created. It would look something like this:
case class Foo[+A : Manifest](a: A) {
def myManifest = implicitly[Manifest[_ <: A]]
def localCopy = copy()
}
val foo = Foo(1)
println(foo.localCopy.myManifest) // Prints "Int"
In the original example, however, it fails in the Bar class because of the second peculiarity: while the type parameters of Bar are known within the Bar class, the type parameters of the type parameters are not. It knows that A in Bar is a Foo or a SubFoo or SubSubFoo, but not if it is a Foo[Int] or a Foo[String]. This is, of course, the well-known type erasure problem in Scala, but it appears as a problem here even when it doesn't seem like the class is doing anything with the type of foos type parameter. But it is, remember there is a secret injection of a manifest every time copy is called, and those manifests overwrite the ones that were there before. Since the Bar class has no idea was the type parameter of foo is, it just creates a manifest of Any and sends that along like this:
def fooCopy = foo.copy()(manifest[Any])
If one has control over the Foo class (e.g. it's not List) then one workaround it by doing all the copying over in the Foo class by adding a method that will do the proper copying, like localCopy above, and return the result:
case class Bar[A <: Foo[Any]](foo: A) {
//def fooCopy = foo.copy()
def fooCopy = foo.localCopy
}
val bar = Bar(Foo(1))
println(bar.fooCopy.myManifest) // Prints "Int"
Another solution is to add Foos type parameter as a manifested type parameter of Bar:
case class Bar[A <: Foo[B], B : Manifest](foo: A) {
def fooCopy = foo.copy()
}
But this scales poorly if class hierarchy is large, (i.e. more members have type parameters, and those classes also have type parameters) since every class would have to have the type parameters of every class below it. It also seems to make the type inference freak out when trying to construct a Bar:
val bar = Bar(Foo(1)) // Does not compile
val bar = Bar[Foo[Int], Int](Foo(1)) // Compiles
There are two problems as you identified. The first problem is the type erasure problem inside of Bar, where Bar doesn't know the type of Foo's manifest. I would personally use the localCopy workaround you suggested.
The second issue is that another implicit is being secretly injected into copy. That issue is resolved by explicitly passing the value again into the copy. For example:
scala> case class Foo[+A](a: A)(implicit val m: Manifest[A #uncheckedVariance])
defined class Foo
scala> case class Bar[A <: Foo[Any]](foo: A) {
| def fooCopy = foo.copy()(foo.m)
| }
defined class Bar
scala> val foo = Foo(1)
foo: Foo[Int] = Foo(1)
scala> val bar = Bar(foo)
bar: Bar[Foo[Int]] = Bar(Foo(1))
scala> bar.fooCopy.m
res2: Manifest[Any] = Int
We see the copy has kept the Int manifest but the type of fooCopy and res2 is Manifest[Any] due to erasure.
Because I needed access to the implicit evidence to do the copy I had to use the explicit implicit (hah) syntax instead of the context bound syntax. But using the explicit syntax caused errors:
scala> case class Foo[+A](a: A)(implicit val m: Manifest[A])
<console>:7: error: covariant type A occurs in invariant position in type => Manifest[A] of value m
case class Foo[+A](a: A)(implicit val m: Manifest[A])
^
scala> case class Foo[+A](a: A)(implicit val m: Manifest[_ <: A])
defined class Foo
scala> val foo = Foo(1)
<console>:9: error: No Manifest available for Int.
WTF? How come the context bound syntax works and the explicit implicit doesn't? I dug around and found a solution to the problem: the #uncheckedVariance annotation.
UPDATE
I dug around some more and found that in Scala 2.10 case classes have been changed to only copy the fields from the first parameter list in copy().
Martin says: case class ness is only bestowed on the first argument
list the rest should not be copied.
See the details of this change at https://issues.scala-lang.org/browse/SI-5009.
Related
What is the difference between the implicit keyword in Scala 2 and given+using in Scala 3? Is it just that implicit has been split up into two keywords, or are the semantics also different, and if so, how?
For the most part, they are the same. However, implicit is no longer used for multiple different concepts. The docs go into more detail, but here's a summary of them:
Using
When declaring parameters, using is just the same as implicit. However, when explicitly passing an implicit argument, you must use using:
def foo(using bar: Bar) = ???
foo(using Bar()) //Cannot use just foo(Bar()) as you would in Scala 2
You can also have implicit by-name parameters in Scala 3.
Given
Givens are also pretty similar to implicit vals/objects/methods.
One nice thing about them is that they can be anonymous, and the compiler will generate a name for them, which looks something like given_F_X_Y if the type of the given were F[X, Y]. More details here.
Another change is that the type of a given must be written explicitly - it cannot be inferred like for an implicit in Scala 2.
A given without parameters maps to an implicit object. given foo: Foo with {...} becomes just implicit object foo extends Foo {...}.
A given with parameters is akin to an implicit def that takes in only more implicit parameters.
given listOrd[T](using ord: Ord[T]): Ord[List[T]] with { ... }
//^^ this maps to this vv
class listOrd[T](implicit ord: Ord[T]) extends Ord[List[T]] { ... }
final implicit def listOrd[T](implicit ord: Ord[T]): listOrd[T] = new listOrd[T]
A given that is merely an alias becomes an implicit def if it is just a reference, or an implicit lazy val otherwise.
val foo: Foo
given Foo = foo
would become final implicit def given_Foo = foo (note the compiler-generated name), but
given foo: Foo = new Foo()
would turn into final implicit lazy val foo: Foo = new Foo() because new Foo() shouldn't be computed unnecessarily.
Instead of using an implicit def for an implicit conversion from A to B, you can now define a given Conversion[A, B] instance.
You can also still use implicit classes in Dotty, but you can directly define extension methods. While methods inside extensions cannot take their own type parameters, they are easier to use than implicit classes.
An additional change in Scala 3 - summon is a method like implicitly, but it can return a type more specific than the one being requested.
Semantics is also different. In Scala 2 Not can be defined with ambiguity trick
trait Not[A]
object Not {
implicit def default[A]: Not[A] = null
implicit def ambig[A](implicit a: A): Not[A] = null
}
implicitly[Not[Int]] // compiles
implicit val s: String = null
// implicitly[Not[String]] // doesn't compile
But in Scala 3 this doesn't work because ambiguity error is not propagated
trait Not[A]
object Not {
given [A]: Not[A] = null
given [A](using a: A): Not[A] = null
// given ambig[A](using a: A): Not[A] = null
}
summon[Not[Int]] // compiles
given String = null
summon[Not[String]] // compiles
One should use scala.util.NotGiven instead
summon[NotGiven[Int]] // compiles
given String = null
// summon[NotGiven[String]] // doesn't compile
(Tested in 3.0.0-M3-bin-20201211-dbc1186-NIGHTLY)
http://dotty.epfl.ch/docs/reference/contextual/givens.html#negated-givens
http://dotty.epfl.ch/docs/reference/changed-features/implicit-resolution.html
I am integrating with some java api, that looks kinda like this (these are java classes, I'll just use scala syntax for brevity:
class AbstractFooBuilder[ActualType <: AbstractFooBuilder, WhatToBuild <: Foo] {
// ...
def withFoo(f: Foo): ActualType
def withBar(b: Bar): ActualType
// ...
def build: WhatToBuild
}
class FooBarBuilder extends AbstractFooBuilder[FooBarBuilder, FooBar]
class FooBazBuilder extends AbstractFooBuilder[FooBazBuilder, FooBaz]
// .. etc
There is a bunch of these, and I am trying to make the creation of these foos less repetitive with something like this:
def anyFoo[T <: Foo, B <: AbstractFooBuilder[B, T] : Manifest](foo: Foo, b: Bar) = manifest
.runtimeClass
.newInstance
.withFoo(foo)
.withBar(bar)
.build
The problem is, that now, to create FooBar, I have to write something like this:
val foobar = new anyFoo[FooBar, FooBarBuilder](foo, bar)
which is longer than I would like. Specifically, once FooBarBuilder type parameter is known, FooBaris the only possibility for the second param ... I wonder if there is some trick I am missing, that would make it possible to "infer" the other param, and only have to specify one.
Any ideas?
Unfortunately, the standard trick to split the type parameters doesn't work here because B depends on T.
But what about getting rid of Manifest and simplifying to
def anyFoo[T <: Foo](builder: AbstractFooBuilder[_, T])(foo: Foo, b: Bar) =
builder.withBar(b).withFoo(foo).build
? You call it as anyFoo(new FooBarBuilder)(foo, b), so T is inferred, solving your problem. And as a bonus, it's faster and there is no runtime error if your class happens not to have a default constructor.
If your method didn't always need to create the builder, you could use by-name parameter or () => AbstractFooBuilder[_, T].
EDIT: Given the comment, this could work:
def mkBuilder[B <: AbstractFooBuilder[B, _] : Manifest]: B = // the complex creation code
anyFoo(mkBuilder[FooBarBuilder])(foo, b) // infers FooBar
The question is whether you can implement mkBuilder without access to T, but if your original code doesn't need a Manifest[T], it should be possible.
Or even
implicit class AnyFoo[T, B <: AbstractFooBuilder[B, T]](builder: B) {
def apply(foo: Foo, b: Bar) = builder.withBar(b).withFoo(foo).build
}
mkBuilder[FooBarBuilder](foo, b)
If I'm reading code correctly. The code does not imply there is only one AbstractFooBuilder factory for this type and this is left to the caller to specify the factory to use.
If you can use implicit, you can do something like this
def anyFoo[T <: Foo](foo: Foo, b: Bar)(implicit factory: AbstractFooBuilder[T]): T = {
factory.
withBar(b).
withFoo(foo).
build
}
I came from C++ world and new to Scala, and this behavior looks unusual.
class G1[A]( val a : A) {
//val c:A = new A //This gives compile error
def fcn1(b: A): Unit = {
//val aobj = new A // This gives compile error
println(a.getClass.getSimpleName)
println(b.getClass.getSimpleName)
}
}
def fcnWithTP[A](): Unit = {
//val a = new A // This gives compile error
//println(a.getClass.getSimpleName)
}
I am not able to crate a object using the type parameter in a class in a function body or a class body. I am only be able to use it in the function parameter.
What is the reason for this? Is this because of type erasure? At run time, the function does not know what the actual type A is, so it cannot create an object of that type?
What is the general rule for this? Does it that mean the type parameter cannot appear in function body or class definition at all? If they can actually appear, what are the examples?
Yes, you're right that this is because of erasure—you don't know anything about A at runtime that you haven't explicitly asserted about it as a constraint in the method signature.
Type erasure on the JVM is only partial, so you can do some horrible things in Scala like ask for the class of a value:
scala> List(1, 2, 3).getClass
res0: Class[_ <: List[Int]] = class scala.collection.immutable.$colon$colon
Once you get to generics, though, everything is erased, so for example you can't tell the following things apart:
scala> List(1, 2, 3).getClass == List("a", "b", "c").getClass
res1: Boolean = true
(In case it's not clear, I think type erasure is unambiguously a good thing, and that the only problem with type erasure on the JVM is that it's not more complete.)
You can write the following:
import scala.reflect.{ ClassTag, classTag }
class G1[A: ClassTag](val a: A) {
val c: A = classTag[A].runtimeClass.newInstance().asInstanceOf[A]
}
And use it like this:
scala> val stringG1: G1[String] = new G1("foo")
stringG1: G1[String] = G1#33d71170
scala> stringG1.c
res2: String = ""
This is a really bad idea, though, since it will crash at runtime for many, many type parameters:
scala> class Foo(i: Int)
defined class Foo
scala> val fooG1: G1[Foo] = new G1(new Foo(0))
java.lang.InstantiationException: Foo
at java.lang.Class.newInstance(Class.java:427)
... 43 elided
Caused by: java.lang.NoSuchMethodException: Foo.<init>()
at java.lang.Class.getConstructor0(Class.java:3082)
at java.lang.Class.newInstance(Class.java:412)
... 43 more
A better approach is to pass in the constructor:
class G1[A](val a: A)(empty: () => A) {
val c: A = empty()
}
And a much better approach is to use a type class:
trait Empty[A] {
def default: A
}
object Empty {
def instance[A](a: => A): Empty[A] = new Empty[A] {
def default: A = a
}
implicit val stringEmpty: Empty[String] = instance("")
implicit val fooEmpty: Empty[Foo] = instance(new Foo(0))
}
class G1[A: Empty](val a: A) {
val c: A = implicitly[Empty[A]].default
}
And then:
scala> val fooG1: G1[Foo] = new G1(new Foo(10101))
fooG1: G1[Foo] = G1#5a34b5bc
scala> fooG1.c
res0: Foo = Foo#571ccdd0
Here we're referring to A in the definition of G1, but we're only making reference to properties and operations that we've confirmed hold or are available at compile time.
Generics are not the same thing as templates. In C++ Foo<Bar> and Foo<Bat> are two different classes, generated at compile time.
In scala or java, Foo[T] is a single class that has with a type parameter. Consider this:
class Foo(val bar)
class Bar[T] {
val foo = new T // if this was possible ...
}
new Bar[Foo]
In C++, (an equivalent of) this would fail to compile, because there is no accessible constructor of Foo that takes no arguments. The compiler would know that when it tried to instantiate a template for Bar<Foo> class, and fail.
In scala, there is no separate class for Bar[Foo], so, at compilation time, the compiler doesn't know anything about T, other than that it is some type. It has no way of knowing whether calling a constructor (or any other method for that matter) is possible or sensible (you can't instantiate a trait for example, or an abstract class), so new T in that context has to fail: it simply does not make sense.
Roughly speaking, you can use type parameters in places where any type can be used (do declare a return type for example, or a variable), but when you are trying to do something that only works for some types, and not for others, you have to make your type param more specific. For example, this: def foo[T](t: T) = t.intValue does not work, but this: def foo[T <: Number](t: T) = t.intValue does.
Well the compiler does not know how to create an instance of type A. You need to either provide a factory function that returns instance of A, or use Manifest which creates instance of A from reflection.
With factory function:
class G1[A](val a:A)(f: () => A) {
val c:A = f()
}
With Manifest:
class G1[A](val a: A)(implicit m: scala.reflect.Manifest[A]) {
val c: A = m.erasure.newInstance.asInstanceOf[A]
}
When using type parameter, usually you will specify more details on the type A, unless you're implementing some sort of container for A that does not directly interact with A. If you need to interact with A, you need some specification on it. You can say A must be a subclass of B
class G1[A <: B](val a : A)
Now compiler would know A is a subclass of B so you can call all functions defined in B on a:A.
I'm reading the book Scala in Depth, chapter 5 about implicits. The author says this on page 102:
The implicit scope used for implicit views is the same as for implicit parameters. But when the compiler is looking for type associations, it uses the type it's attempting to convert from [my emphasis], not the type it's attempting to convert to.
And yet, a few pages later he shows an example, with a complexmath.ComplexNumber class. You import i, which is a ComplexNumber, and call it's * method, which takes a ComplexNumber argument.
import complexmath.i
i * 1.0
To convert 1.0 into a ComplexNumber, this finds an implicit conversion that was defined like so:
package object complexmath {
implicit def realToComplex(r: Double) = new ComplexNumber(r, 0)
val i = ComplexNumber(0, 1)
But that contradicts the first statement, no? It needed to find Double => ComplexNumber. Why did it look in the complexmath package, which is part of the implicit scope for ComplexNumber but not for Double?
The spec says about views:
the implicit scope is the one of T => pt.
i.e., Function[T, pt], so implicit scope includes the classes associated with both T and pt, the source and target of the conversion.
scala> :pa
// Entering paste mode (ctrl-D to finish)
class B
class A
object A { implicit def x(a: A): B = new B }
// Exiting paste mode, now interpreting.
warning: there were 1 feature warning(s); re-run with -feature for details
defined class B
defined class A
defined object A
scala> val b: B = new A
b: B = B#63b41a65
scala> def f(b: B) = 3 ; def g = f(new A)
f: (b: B)Int
g: Int
scala> :pa
// Entering paste mode (ctrl-D to finish)
class A
class B
object B { implicit def x(a: A): B = new B }
// Exiting paste mode, now interpreting.
scala> val b: B = new A
b: B = B#6ba3b481
I think you are misunderstanding his text.
The compiler will look for the implicit conversion in all available scopes until it finds a suitable one.
In the example you specified it'll find one being provided by the complexmath package.
However, is the definition of 'suitable one' that we are interested here. In case of implicit conversions, the compiler will look for a conversion from *Double* to the expected type ComplexNumber.
In other words, it'll inspect all conversions from *Double* until it finds one that can convert a Double to the target type.
Josh, the author, is not saying that the compiler needs a conversion defined in an object associated with the Double object. The conversion can be defined everywhere.
In this particular case, the conversion is defined in the package object associated with the ComplexNumber object. And that's normal, is the ComplexNumber object that 'wants' to be compatible with Double.
And since the usage implies the import of ComplexNumber and therefore the import of package 'complexmath', the conversion will be always in scope.
So this is more about what the compiler already *know*s about an expression.
You have in the example:
import complexmath.i
i * 1.0
The compiler looks at this to start with:
1) I have a type for i, it is Complex
2) I have a method on i called *, it takes an argument of type Complex
3) You passed me an Int, but I need a Complex. let's see if i have an implicit which gives me that.
This example works because the * method is defined on Complex.
Hope that helps!
Either source or target works:
object Foo {
implicit def bar(b: Bar): Foo = new Foo {}
implicit def foo(f: Foo): Bar = new Bar {}
}
trait Foo
trait Bar
implicitly[Foo => Bar] // ok
implicitly[Bar => Foo] // ok
val b = new Bar {}
val bf: Foo = b // ok
val f = new Foo {}
val fb: Bar = f // ok
So I think that sentence is wrong (?)
I would like to define a method parameterized with type T that has behavior dependent on what implicit argument can be found of type Box[T]. The following code has this method defined as foo. When called with foo[Int] or foo[String] it will without issue return 1 or "two" as expected.
Where things get weird is with the method bar. It is defined as returning an Int, but instead of foo[Int] I have just foo. My hope was that the compiler would infer that T must be of type Int. It does not do that and instead fails:
bash $ scalac Code.scala
Types.scala:15: error: ambiguous implicit values:
both value one in object Main of type => Main.Box[Int]
and value two in object Main of type => Main.Box[java.lang.String]
match expected type Main.Box[T]
def bar: Int = foo
^
one error found
What is causing this error? Replacing foo with foo[Int] compiles fine. The simpler situation where there is no Box[T] type also compiles fine. That example is also below and uses argle and bargle instead of foo and bar.
object Main extends Application {
case class Box[T](value: T)
implicit val one = Box(1)
implicit val two = Box("two")
def foo[T](implicit x: Box[T]): T = {
x.value
}
// does not compile:
// def bar: Int = foo
// does compile
def bar: Int = foo[Int]
println(bar)
// prints 1
// the simpler situation where there is no Box type
implicit val three = 3
implicit val four = "four"
def argle[T](implicit x: T): T = x
def bargle: String = argle
println(bargle)
// prints "four"
}
What is going on in this snippet that causes this behavior? What about this interaction of implicit arguments, type inference, and erasure is causing problems? Is there a way to modify this code such that the line def foo: Int = bar works?
Someone else will have to explain why the type inference mechanism cannot handle that case, but if you are looking to cleanup your code you could probably do this:
object Test extends App {
case class Box[T](value: T)
implicit val one: Box[Int] = Box(1)
implicit val two: Box[String] = Box("two")
def foo[T : Box]: T = implicitly[Box[T]].value
val bar = foo[Int]
}
Note that:
I removed the type annotation from bar so you are really just indicating the type once (just in a different spot than you wanted)
I am using App instead of deprecated Application
Using a context bound in the type signature of foo
This might be related to SI-3346, though there it is implicit arguments to implicit conversions, and here you have a single implicit.