How does Scala use explicit types when resolving implicits? - scala

I have the following code which uses spray-json to deserialise some JSON into a case class, via the parseJson method.
Depending on where the implicit JsonFormat[MyCaseClass] is defined (in-line or imported from companion object), and whether there is an explicit type provided when it is defined, the code may not compile.
I don't understand why importing the implicit from the companion object requires it to have an explicit type when it is defined, but if I put it inline, this is not the case?
Interestingly, IntelliJ correctly locates the implicit parameters (via cmd-shift-p) in all cases.
I'm using Scala 2.11.7.
Broken Code - Wildcard import from companion object, inferred type:
import SampleApp._
import spray.json._
class SampleApp {
import MyJsonProtocol._
val inputJson = """{"children":["a", "b", "c"]}"""
println(s"Deserialise: ${inputJson.parseJson.convertTo[MyCaseClass]}")
}
object SampleApp {
case class MyCaseClass(children: List[String])
object MyJsonProtocol extends DefaultJsonProtocol {
implicit val myCaseClassSchemaFormat = jsonFormat1(MyCaseClass)
}
}
Results in:
Cannot find JsonReader or JsonFormat type class for SampleAppObject.MyCaseClass
Note that the same thing happens with an explicit import of the myCaseClassSchemaFormat implicit.
Working Code #1 - Wildcard import from companion object, explicit type:
Adding an explicit type to the JsonFormat in the companion object causes the code to compile:
import SampleApp._
import spray.json._
class SampleApp {
import MyJsonProtocol._
val inputJson = """{"children":["a", "b", "c"]}"""
println(s"Deserialise: ${inputJson.parseJson.convertTo[MyCaseClass]}")
}
object SampleApp {
case class MyCaseClass(children: List[String])
object MyJsonProtocol extends DefaultJsonProtocol {
//Explicit type added here now
implicit val myCaseClassSchemaFormat: JsonFormat[MyCaseClass] = jsonFormat1(MyCaseClass)
}
}
Working Code #2 - Implicits inline, inferred type:
However, putting the implicit parameters in-line where they are used, without the explicit type, also works!
import SampleApp._
import spray.json._
class SampleApp {
import DefaultJsonProtocol._
//Now in-line custom JsonFormat rather than imported
implicit val myCaseClassSchemaFormat = jsonFormat1(MyCaseClass)
val inputJson = """{"children":["a", "b", "c"]}"""
println(s"Deserialise: ${inputJson.parseJson.convertTo[MyCaseClass]}")
}
object SampleApp {
case class MyCaseClass(children: List[String])
}

After searching for the error message Huw mentioned in his comment, I was able to find this StackOverflow question from 2010: Why does this explicit call of a Scala method allow it to be implicitly resolved?
This led me to this Scala issue created in 2008, and closed in 2011: https://issues.scala-lang.org/browse/SI-801 ('require explicit result type for implicit conversions?')
Martin stated:
I have implemented a slightly more permissive rule: An implicit conversion without explicit result type is visible only in the text following its own definition. That way, we avoid the cyclic reference errors. I close for now, to see how this works. If we still have issues we migth come back to this.
This holds - if I re-order the breaking code so that the companion object is declared first, then the code compiles. (It's still a little weird!)
(I suspect I don't see the 'implicit method is not applicable here' message because I have an implicit value rather than a conversion - though I'm assuming here that the root cause is the same as the above).

Related

Providing implicit evidence for context bounds on Object

I'm trying to write some abstractions in some Spark Scala code, but running into some issues when using objects. I'm using Spark's Encoder which is used to convert case classes to database schema's here as an example, but I think this question goes for any context bound.
Here is a minimal code example of what I'm trying to do:
package com.sample.myexample
import org.apache.spark.sql.Encoder
import scala.reflect.runtime.universe.TypeTag
case class MySparkSchema(id: String, value: Double)
abstract class MyTrait[T: TypeTag: Encoder]
object MyObject extends MyTrait[MySparkSchema]
Which fails with the following compilation error:
Unable to find encoder for type com.sample.myexample.MySparkSchema. An implicit Encoder[com.sample.myexample.MySparkSchema] is needed to store com.sample.myexample.MySparkSchema instances in a Dataset. Primitive types (Int, String, etc) and Product types (case classes) are supported by importing spark.implicits._ Support for serializing other types will be added in future releases.
I tried defining the implicit evidence in the object like such: (the import statement was suggested by IntelliJ, but it looks a bit weird)
import com.sample.myexample.MyObject.encoder
object MyObject extends MyTrait[MySparkSchema] {
implicit val encoder: Encoder[MySparkSchema] = Encoders.product[MySparkSchema]
}
Which fails with the error message
MyTrait.scala:13:25: super constructor cannot be passed a self reference unless parameter is declared by-name
One other thing I tried is to convert the object to a class and provide implicit evidence to the constructor:
class MyObject(implicit evidence: Encoder[MySparkSchema]) extends MyTrait[MySparkSchema]
This compiles and works fine, but at the expense of MyObject now being a class instead.
Question: Is it possible to provide implicit evidence for the context bounds when extending a trait? Or does the implicit evidence force me to make a constructor and use class instead?
Your first error almost gives you the solution, you have to import spark.implicits._ for Product types.
You could do this:
val spark: SparkSession = SparkSession.builder().getOrCreate()
import spark.implicits._
Full Example
package com.sample.myexample
import org.apache.spark.sql.Encoder
import scala.reflect.runtime.universe.TypeTag
case class MySparkSchema(id: String, value: Double)
abstract class MyTrait[T: TypeTag: Encoder]
val spark: SparkSession = SparkSession.builder().getOrCreate()
import spark.implicits._
object MyObject extends MyTrait[MySparkSchema]

In scala 2.13, why implicit scope of a companion object may sometimes be misaligned? How to correct it?

The following is a simple example that tests the implicit feature of the scala 2.13 compiler:
object OverridingScope {
trait System {
trait Handler[T]
}
object Sys1 extends System {
object Handler {
implicit def handle1: Handler[Int] = new Handler[Int] {}
}
implicit def handle2: Handler[Long] = new Handler[Long] {}
}
}
According this article:
Where does Scala look for implicits?
The companion object of the trait Handler can be automatically imported as part of the implicit scope when type Handler[_] is used, the outer object Sys1 should be irrelevant. This means handle1 should be visible and handle2 should be hidden.
But the following test case shows the exact opposite:
class OverridingScope extends BaseSpec {
import OverridingScope._
it("implicits in companion is in scope") {
// val hh = implicitly[Sys1.Handler[Int]]
// Doesn't work
}
it("implicits in outer object is in scope") {
val hh = implicitly[Sys1.Handler[Long]]
// Should NOT work
}
}
Considering that Handler is both a class & object name, the fact that handle1 doesn't work looks highly suspicious. Also, I've checked the full list of implicit import:
First look in current scope
Implicits defined in current scope
Explicit imports
wildcard imports
Same scope in other files
Now look at associated types in
Companion objects of a type
Implicit scope of an argument's type (2.9.1)
Implicit scope of type arguments (2.8.0)
Outer objects for nested types
Other dimensions
None of the above can explain why handle2 is in the scope. What could be the reasons?
BTW: For those who dismissed implicit as a important scala feature: Implicit is the only way to implement type class, and if you want to implement it with minimal boilerplate, sometimes the above pattern can be the only choice.
Point 3 here explains that implicits/givens in prefixes of the type in question contribute to its implicit scope (in Scala 3 not if it's a package). So for the type Sys1.Handler[Long] implicits in Sys1 are in the implicit scope because Sys1 is a non-package prefix.
handle1 is not in the implicit scope because object Handler in object Sys1 is not the companion object of trait Handler in trait System.

Importing generic implicits from class instances

I'm trying to make a generic implicit provider which can create an implicit value for a given type, something in the lines of:
trait Evidence[T]
class ImplicitProvider[T] {
class Implementation extends Evidence[T]
implicit val evidence: Evidence[T] = new Implementation
}
To use this implicit, I create a val provider = new ImplicitProvider[T] instance where necessary and import from it import provider._. This works fine as long as there is just one instance. However sometimes implicits for several types are needed in one place
case class A()
case class B()
class Test extends App {
val aProvider = new ImplicitProvider[A]
val bProvider = new ImplicitProvider[B]
import aProvider._
import bProvider._
val a = implicitly[Evidence[A]]
val b = implicitly[Evidence[B]]
}
And this fails to compile with could not find implicit value for parameter and not enough arguments for method implicitly errors.
If I use implicit vals from providers directly, everything starts to work again.
implicit val aEvidence = aProvider.evidence
implicit val bEvidence = bProvider.evidence
However I'm trying to avoid importing individual values, as there are actually several implicits inside each provider and the goal is to abstract them if possible.
Can this be achieved somehow or do I want too much from the compiler?
The issue is that when you import from both objects, you're bringing in two entities that have colliding names: evidence in aProvider and evidence in bProvider. The compiler cannot disambiguate those, both because of how its implemented, and because it'd be a bad idea for implicits, which can already be arcane, to be able to do things that cannot be done explicitly (disambiguating between clashing names).
What I don't understand is what the point of ImplicitProvider is. You can pull the Implementation class out to the top level and have an object somewhere that holds the implicit vals.
class Implementation[T] extends Evidence[T]
object Evidence {
implicit val aEvidence: Evidence[A] = new Implementation[A]
implicit val bEvidence: Evidence[B] = new Implementation[B]
}
// Usage:
import Evidence._
implicitly[Evidence[A]]
implicitly[Evidence[B]]
Now, there is no name clash.
If you need to have an actual ImplicitProvider, you can instead do this:
class ImplicitProvider[T] { ... }
object ImplicitProviders {
implicit val aProvider = new ImplicitProvider[A]
implicit val bProvider = new ImplicitProvider[B]
implicit def ImplicitProvider2Evidence[T: ImplicitProvider]: Evidence[T]
= implicitly[ImplicitProvider[T]].evidence
}
// Usage
import ImplicitProviders._
// ...

Scala shapeless implicit resolution StackOverflowError with ArgonautShapeless

Im using ArgonautShapeless to define some json codecs.
When I provide the type for my codec I get StackOverflowError's but If I leave the type off it works. How can I provide the type?
My understanding of the problem is that the implicit lookup from def of[A: DecodeJson] = implicitly[DecodeJson[A]] finds my definition on the same line implicit def fooCodec: DecodeJson[Foo] and thus is recursive so breaks.
Is there some other way that will allow me to provide the type? Ideally I want to have one object in my project where I define all of the codes and they may depend on each other.
import $ivy.`com.github.alexarchambault::argonaut-shapeless_6.2:1.2.0-M4`
import argonaut._, Argonaut._
case class Foo(a: Int)
object SomeCodecs {
import ArgonautShapeless._
// this doesnt work
implicit def fooCodec: DecodeJson[Foo] = DecodeJson.of[Foo]
}
import SomeCodecs._
"""{"a":1}""".decode[Foo]
java.lang.StackOverflowError
ammonite.$sess.cmd3$SomeCodecs$.fooCodec(cmd3.sc:3)
It works if I leave the type off.
object SomeCodecs {
import ArgonautShapeless._
// this works
implicit def fooCodec = DecodeJson.of[Foo]
}
import SomeCodecs._
"""{"a":1}""".decode[Foo]
res4: Either[Either[String, (String, CursorHistory)], Foo] = Right(Foo(1))
Thanks

Scala: Use implicits in companion object

I am creating companion object in scala and trying to use object implictis functions in class without import. But whenever, trying to compile the code I am getting an error: type mismatch; seems it is not able to import implictis automatically. Following is my code:
object ImplicitTest5 {
implicit def dollarToRupa(dollar: Dollar): Rupa = {
println("calling .... dollarToEuro")
Rupa(dollar.value)
}
implicit def dollarToEuro(dollar: Dollar): Euro = {
println("calling .... dollarToEuro")
Euro(dollar.value)
}
}
case class Dollar(value: Double)
case class Euro(value: Double)
case class Rupa(value: Double)
class ImplicitTest5 {
private val value = "String"
def conversion = {
val euro: Euro = Dollar(3.1)
println(s" ----- $euro")
}
}
When i am using import ImplicitTest5._ in my class, it will compile and run fine. According to Programming in Scala, Page: 478 it will be working fine, and there is no need to define import.
In this case, the conversion dollarToEuro is said to be associated to
the type Dollar. The compiler will find such an associated conversion
every time it needs to convert from an instance of type Dollar.
There’s no need to import the conversion separately into your program.
Something is going wrong with my sample or my understandings are misleading ?
Something is going wrong with my sample or my understandings are
misleading
The conversion to Dollar will be associated with it if you define it inside Dollars companion object. Currently, all your implicits are defined on ImplicitTest5, which isn't where the compiler looks for implicits in regards to the Dollar class. This forces you to explictly import the object containing those implicits. Instead, do:
case class Dollar(value: Double)
object Dollar {
implicit def dollarToEuro(dollar: Dollar): Euro = {
println("calling .... dollarToEuro")
Euro(dollar.value)
}
}
For more, see "Where does Scala look for implicits"
By default scala compiler will look into the companion object of source and target objects. So if for example you are converting from Dollor to Euro, the implicit method can be in Dollor or Euro companion object. The compiler will pick it automatically. Otherwise you have to bring it explicitly in the scope.