I am trying to abstract out the json parsing logic that gets triggered for a specific type.
I started out creating a Parser trait as follows:
trait Parser {
def parse[T](payload : String) : Try[T]
}
I have an implementation of this trait called JsonParser which is:
class JsonParser extends Parser {
override def parse[T](payload: String): Try[T] = parseInternal(payload)
private def parseInternal[T:JsonParserLike](payload:String):Try[T] = {
implicitly[JsonParserLike[T]].parse(payload)
}
}
The JsonParserLike is defined as follows:
trait JsonParserLike[T] {
def parse(payload: String): Try[T]
}
object JsonParserLike {
implicit val type1Parser:JsonParserLike[Type1] = new JsonParserLike[Type1]
{
//json parsing logic for Type1
}
implicit val type2Parser:JsonParserLike[Type2] = new JsonParserLike[Type2]
{
//json parsing logic for Type2
}
}
When I try compiling the above, the compilation fails with:
ambiguous implicit values:
[error] both value type1Parse in object JsonParserLike of type => parser.jsonutil.JsonParserLike[parser.models.Type1]
[error] and value type2Parser in object JsonParserLike of type => parser.jsonutil.JsonParserLike[parser.models.Type2]
[error] match expected type parser.jsonutil.JsonParserLike[T]
[error] override def parse[T](payload: String): Try[T] = parseInternal(payload)
Not sure why the implicit resolution is failing here. Is it because the parse method in the Parser trait doesn't have an argument of type parameter T?
I tried another approach as follows:
trait Parser {
def parse[T](payload : String) : Try[T]
}
class JsonParser extends Parser {
override def parse[T](payload: String): Try[T] = {
import workflow.parser.JsonParserLike._
parseInternal[T](payload)
}
private def parseInternal[U](payload:String)(implicit c:JsonParserLike[U]):Try[U] = {
c.parse(payload)
}
}
The above gives me the following error:
could not find implicit value for parameter c: parser.JsonParserLike[T]
[error] parseInternal[T](payload)
[error]
^
Edit: Adding the session from the REPL
scala> case class Type1(name: String)
defined class Type1
scala> case class Type2(name:String)
defined class Type2
scala> :paste
// Entering paste mode (ctrl-D to finish)
import scala.util.{Failure, Success, Try}
trait JsonParserLike[+T] {
def parse(payload: String): Try[T]
}
object JsonParserLike {
implicit val type1Parser:JsonParserLike[Type1] = new JsonParserLike[Type1] {
override def parse(payload: String): Try[Type1] = Success(Type1("type1"))
}
implicit val type2Parser:JsonParserLike[Type2] = new JsonParserLike[Type2] {
override def parse(payload: String): Try[Type2] = Success(Type2("type2"))
}
}
// Exiting paste mode, now interpreting.
import scala.util.{Failure, Success, Try}
defined trait JsonParserLike
defined object JsonParserLike
scala> :paste
// Entering paste mode (ctrl-D to finish)
trait Parser {
def parse[T](payload : String) : Try[T]
}
class JsonParser extends Parser {
override def parse[T](payload: String): Try[T] = parseInternal(payload)
private def parseInternal[T:JsonParserLike](payload:String):Try[T] = {
implicitly[JsonParserLike[T]].parse(payload)
}
}
// Exiting paste mode, now interpreting.
<pastie>:24: error: ambiguous implicit values:
both value type1Parser in object JsonParserLike of type => JsonParserLike[Type1]
and value type2Parser in object JsonParserLike of type => JsonParserLike[Type2]
match expected type JsonParserLike[T]
override def parse[T](payload: String): Try[T] = parseInternal(payload)
As I've already tried to explain in the comments, the problem is that the method
override def parse[T](payload: String): Try[T] = parseInternal(payload)
does not accept any JsonParserLike[T] instances. Therefore, the compiler has no way to insert the right instance of JsonParserLike[T] at the call site (where the type T is known).
To make it work, one would have to add some kind of token that uniquely identifies type T to the argument list of parse. One crude way would be to add a JsonParserLike[T] itself:
import util.Try
trait Parser {
def parse[T: JsonParserLike](payload : String) : Try[T]
}
class JsonParser extends Parser {
override def parse[T: JsonParserLike](payload: String): Try[T] =
parseInternal(payload)
private def parseInternal[T:JsonParserLike](payload:String):Try[T] = {
implicitly[JsonParserLike[T]].parse(payload)
}
}
trait JsonParserLike[T] {
def parse(payload: String): Try[T]
}
object JsonParserLike {
implicit val type1Parser: JsonParserLike[String] = ???
implicit val type2Parser: JsonParserLike[Int] = ???
}
Now it compiles, because the JsonParserLike[T] required by parseInternal is inserted automatically as an implicit parameter to parse.
This might be not exactly what you want, because it creates a hard dependency between Parser interface and the JsonParserLike typeclass. You might want to get some inspiration from something like shapeless.Typeable to get rid of the JsonParserLike in the Parser interface, or just rely on circe right away.
It seems like there is an extra complexity from a mix of different types of polymorphism in both examples. Here is a minimal example of just a type class:
// type class itself
trait JsonParser[T] {
def parse(payload: String): Try[T]
}
// type class instances
object JsonParserInstances {
implicit val type1Parser: JsonParser[Type1] = new JsonParser[Type1] {
def parse(payload: String): Try[Type1] = ???
}
implicit val type2Parser: JsonParser[Type2] = new JsonParser[Type2] {
def parse(payload: String): Try[Type2] = ???
}
}
// type class interface
object JsonInterface {
def parse[T](payload: String)(implicit T: JsonParser[T]): Try[T] = {
T.parse(payload)
}
}
def main(args: Array[String]): Unit = {
import JsonParserInstances._
JsonInterface.parse[Type1]("3")
JsonInterface.parse[Type2]("3")
}
More info:
Chapter on Typeclasses in free Scala with Cats Book
Related
I'm new to Scala and still learning about most of the features.
I'm trying to convert a Dataset[T] to Json.
I'm using json play to create implicit Writes.
Type parameters work fine when I load my Dataset:
def processEvent[T](spark: SparkSession, inputPath: String)(implicit encoder: Encoder[T]): Unit = {
val ds = spark.read.parquet(inputPath).as[T]
ds.collect().foreach { event =>
val serializedEvent = eventToJson[T](spark, event)
postEvent(serializedEvent)
}
}
But in the call to EventToJson, I get the error: No Json serializer found for type T. Try to implement an implicit Writes or Format for this type.
def eventToJson[T](spark: SparkSession, event: T): String = {
Json.toJson(event).toString()
}
When I replace the parameterized type with one of my case classes, the code works fine:
def eventToJson(spark: SparkSession, event: MyCaseClass): String = {
Json.toJson(event).toString()
}
Why doesn't the parameterized type find the corresponding case class and implicit Writes?
Try to constrain T with a context bound:
def eventToJson[T: Writes](spark: SparkSession, event: T): String = {
Json.toJson(event).toString()
}
or
def eventToJson[T: Format](spark: SparkSession, event: T): String = {
Json.toJson(event).toString()
}
Then probably you'll have to add the same context bound to processEvent
import org.apache.spark.sql.{Encoder, SparkSession}
import play.api.libs.json.{Json, Writes}
def processEvent[T: Encoder: Writes](spark: SparkSession, inputPath: String): Unit = {
val ds = spark.read.parquet(inputPath).as[T]
ds.collect().foreach { event =>
val serializedEvent = eventToJson[T](spark, event)
postEvent(serializedEvent)
}
}
def eventToJson[T: Writes](spark: SparkSession, event: T): String = {
Json.toJson(event).toString()
}
Why doesn't the parameterized type find the corresponding case class and implicit Writes?
Because it is defined for the case class, not for T (even though T can become the type of case class when you apply eventToJson).
In Scala if a definition of method foo produces compile error that an implicit of type Bar is not found then you can try to add that implicit to the signature of foo
def foo(implicit bar: Bar) = ???
In this way you postpone resolution of implicit till the time when foo is called.
If the type is parametric
def foo[T](implicit bar: Bar[T]) = ???
this can be rewritten with a context bound
def foo[T: Bar] = ???
Typeclasses methods called in functions with type parameters
How to resolve implicit lookup by bounded generic?
Implicit Encoder for TypedDataset and Type Bounds in Scala
I have a trait for which I wanted to write Typeclasses for. This trait actually is a contract to do JSON to Case class conversion and vice versa. The definition of the trait is as below:
trait Converter[T] {
def convertFromJson(msg: String): Either[ConverterError, T]
def convertToJson(msg: T): String
}
Now, for one of the case classes that I have, I have defined the implementation like this:
object Converter extends DefaultJsonProtocol {
implicit object DefaultMessageConversions extends Converter[DefaultMessage] {
implicit val json = jsonFormat(DefaultMessage, "timestamp")
def convertFromJson(msg: String): Either[ConverterError, DefaultMessage] = {
try {
Right(msg.parseJson.convertTo[DefaultMessage])
}
catch {
case _: Exception => Left(ConverterError("Shit happens"))
}
}
def convertToJson(msg: DefaultMessage) = {
implicit val writes = Json.writes[DefaultMessage]
Json.toJson(msg).toString
}
}
def apply[T: Converter] : Converter[T] = implicitly
}
But I run into some compiler errors when I tried to build my project. I'm not sure what I did wrong?
[error] /Users/joesan/ingestion/src/main/scala/com/my/project/common/JsonMessageConversion.scala:28: could not find implicit value for evidence parameter of type com.my.project.common.JsonMessageConversion.Converter.JF[org.joda.time.DateTime]
[error] implicit val json = jsonFormat(DefaultMessage, "timestamp")
Here is how my case class look like:
case class DefaultMessage(timestamp: DateTime) extends KafkaMessage {
def this() = this(DateTime.now(DateTimeZone.UTC))
}
Your DefaultMessage uses org.joda.time.DateTime and spray-json does not know how to serialize/deserialize it out of the box.
Therefore you need to define a RootJsonFormat[DateTime] and bring it in implicit scope.
Here is an example implementation.
(Scala 2.11.8)
Consider the following code:
object ScalaTest extends App {
class Wrapper {
import Wrapper._
def init(): Unit = {
// "could not find implicit value for parameter tc: ScalaTest.Wrapper.TC[Int]"
printWithTC(123)
// Compiles
printWithTC(123)(IntTC)
// Compiles again!
printWithTC(132)
}
}
object Wrapper {
trait TC[A] {
def text(a: A): String
}
implicit object IntTC extends TC[Int] {
override def text(a: Int) = s"int($a)"
}
def printWithTC[A](a: A)(implicit tc: TC[A]): Unit = {
println(tc.text(a))
}
}
(new Wrapper).init()
}
I have a bunch of questions regarding this piece of code:
Why doesn't IntTC get resolved in the first place?
Why it compiles after being used once? (if you comment out the first invocation, code works)
Where should typeclass implicits be placed to get resolved properly?
Use a val with a explicit return type. See https://github.com/scala/bug/issues/801 and https://github.com/scala/bug/issues/8697 (among others).
Implicit objects have the same issue as implicit vals and defs with inferred return types. As for your second question: when IntTC is used explicitly you force the compiler to typecheck it, so after that point its type is known and can be found by implicit search.
class Wrapper {
import Wrapper._
def init(): Unit = {
// Compiles
printWithTC(123)
// Compiles
printWithTC(123)(IntTC)
// Compiles
printWithTC(132)
}
}
object Wrapper {
trait TC[A] {
def text(a: A): String
}
implicit val IntTC: TC[Int] = new TC[Int] {
override def text(a: Int) = s"int($a)"
}
def printWithTC[A](a: A)(implicit tc: TC[A]): Unit = {
println(tc.text(a))
}
}
If you really want your implicit to be evaluated lazily like an object, you can use an implicit lazy val with an explicit type.
Define the implicit before the use of it.
object Wrapper {
trait TC[A] {
def text(a: A): String
}
implicit object IntTC extends TC[Int] {
override def text(a: Int) = s"int($a)"
}
def printWithTC[A](a: A)(implicit tc: TC[A]): Unit = {
println(tc.text(a))
}
}
class Wrapper {
import Wrapper._
def init(): Unit = {
// "could not find implicit value for parameter tc: ScalaTest.Wrapper.TC[Int]"
printWithTC(123)
// Compiles
printWithTC(123)(IntTC)
// Compiles again!
printWithTC(132)
}
}
(new Wrapper).init()
Scala 2.11.4
I have the following trait
trait MyTrait{
def met(msg: String): Unit
}
and class
class MyClass{
def ac(mt: MyTrait) = {
//do some job
}
}
Now, I want to invoke ac as follows:
val myClass = new MyClass
myClass.as((msg: String) => println(msg)) //meaning that just print out Strings to stdout
So, I tried to add implicit conversion to MyTrait:
trait MyTrait {
def met(msgString: String): Unit
implicit def function2MyTrait(f: String => Unit): MyTrait = new MyTraitImpl(f)
private[this] class MyTraitImpl(f: String => Unit) extends MyTrait{
override def met(msgString: String): Unit = f(msgString)
}
}
but it refuses to compile:
Error:(16, 89) type mismatch;
found : String => Unit
required: com.test.trace.MyTrait
I came from Java 8. Is there a way to do such a thing in Scala?
function2MyTrait needs to live in the companion object:
trait MyTrait {
def met(msgString: String): Unit
}
object MyTrait {
implicit def function2MyTrait(f: String => Unit): MyTrait = new MyTraitImpl(f)
private[this] class MyTraitImpl(f: String => Unit) extends MyTrait{
override def met(msgString: String): Unit = f(msgString)
}
}
This will be no longer be necessary in Scala 2.12, as it'll allow using lambdas to implement MyTrait without going through String => Unit.
I'm trying to add an implicit value to (what I believe is) the companion object of a case class, but this implicit value is not found.
I'm trying to achieve something like the following:
package mypackage
object Main {
def main(args: Array[String]): Unit = {
val caseClassInstance = MyCaseClass("string")
val out: DataOutput = ...
serialize(out, caseClassInstance)
// the above line makes the compiler complain that there is no
// Serializer[MyCaseClass] in scope
}
def serialize[T : Serializer](out: DataOutput, t: T): Unit = {
...
}
}
object MyCaseClass {
// implicits aren't found here
implicit val serializer: Serializer[MyCaseClase] = ...
}
case class MyCaseClass(s: String) {
// some other methods
}
I've explicitly added the package here to show that both the MyCaseClass case class and object should be in scope. I know that the object is actually being constructed because I can get this to compile if I add
implicit val serializer = MyCaseClass.serializer
to main (though notably not if I add import MyCaseClass.serializer).
I'm concerned that the MyCaseClass object is not actually a companion of the case class, because if I explicitly define apply and unapply on the object and then attempt to call MyCaseClass.apply("string") in main, the compiler gives the following error:
ambiguous reference to overloaded definition,
both method apply in object MyCaseClass of type (s: String)mypackage.MyCaseClass
and method apply in object MyCaseClass of type (s: String)mypackage.MyCaseClass
match argument types (String)
val a = InputRecord.apply("string")
^
If it's not possible to take this approach, is there a way to use type classes with case classes without creating an implicit value every time it must be brought into scope?
EDIT: I'm using scala 2.10.3.
EDIT 2: Here's the example fleshed out:
package mypackage
import java.io.{DataOutput, DataOutputStream}
object Main {
def main(args: Array[String]): Unit = {
val caseClassInstance = MyCaseClass("string")
val out: DataOutput = new DataOutputStream(System.out)
serialize(out, caseClassInstance)
// the above line makes the compiler complain that there is no
// Serializer[MyCaseClass] in scope
}
def serialize[T : Serializer](out: DataOutput, t: T): Unit = {
implicitly[Serializer[T]].write(out, t)
}
}
object MyCaseClass {
// implicits aren't found here
implicit val serializer: Serializer[MyCaseClass] = new Serializer[MyCaseClass] {
override def write(out: DataOutput, t: MyCaseClass): Unit = {
out.writeUTF(t.s)
}
}
}
case class MyCaseClass(s: String) {
// some other methods
}
trait Serializer[T] {
def write(out: DataOutput, t: T): Unit
}
This actually compiles, though. I am getting this issue when using Scoobi's WireFormat[T] instead of Serializer, but can't provide a concise, runnable example due to complexity and the Scoobi dependency. I will try to create a more relevant example, but it seems as though the issue is not as general as I thought.
It turns out that the type class instances actually need to be implicit values, rather than objects. The MyCaseClass object above works because its serializer is assigned to an implicit value. However, this implementation
object MyCaseClass {
implicit object MyCaseClassSerializer extends Serializer[MyCaseClass] {
override def write(out: DataOutput, t: MyCaseClass): Unit = {
out.writeUTF(t.s)
}
}
}
fails with the error
Main.scala:9: error: could not find implicit value for evidence parameter of type mypackage.Serializer[mypackage.MyCaseClass]
serialize(out, caseClassInstance)
^
In my real code, I was using an auxiliary function to generate the Serializer[T] (see https://github.com/NICTA/scoobi/blob/24f48008b193f4e87b9ec04d5c8736ce0725d006/src/main/scala/com/nicta/scoobi/core/WireFormat.scala#L137). Despite the function having its own explicit return type, the type of the assigned value was not being inferred correctly by the compiler.
Below is the full example from the question with such a Serializer-generator.
package mypackage
import java.io.{DataOutput, DataOutputStream}
object Main {
import Serializer._
def main(args: Array[String]): Unit = {
val caseClassInstance = MyCaseClass("string")
val out: DataOutput = new DataOutputStream(System.out)
serialize(out, caseClassInstance)
}
def serialize[T : Serializer](out: DataOutput, t: T): Unit = {
implicitly[Serializer[T]].write(out, t)
}
}
object MyCaseClass {
import Serializer._
// does not compile without Serializer[MyCaseClass] type annotation
implicit val serializer: Serializer[MyCaseClass] =
mkCaseSerializer(MyCaseClass.apply _, MyCaseClass.unapply _)
}
case class MyCaseClass(s: String)
trait Serializer[T] {
def write(out: DataOutput, t: T): Unit
}
object Serializer {
// does not compile without Serializer[String] type annotation
implicit val stringSerializer: Serializer[String] = new Serializer[String] {
override def write(out: DataOutput, s: String): Unit = {
out.writeUTF(s)
}
}
class CaseClassSerializer[T, A : Serializer](
apply: A => T, unapply: T => Option[A]) extends Serializer[T] {
override def write(out: DataOutput, t: T): Unit = {
implicitly[Serializer[A]].write(out, unapply(t).get)
}
}
def mkCaseSerializer[T, A : Serializer]
(apply: A => T, unapply: T => Option[A]): Serializer[T] =
new CaseClassSerializer(apply, unapply)
}
This related, simple code below prints 1.
object A{
implicit def A2Int(a:A)=a.i1
}
case class A(i1:Int,i2:Int)
object Run extends App{
val a=A(1,2)
val i:Int=a
println(i)
}