I'm trying to write a nice generic persistence service with one implementation for mongo using Reactive Mongo and I'm struggling with providing the implicit reader/writer for my record class children. Here's the code.
First the base record class (each persisted record must implement it.
abstract class Record {
val _id: BSONObjectID = BSONObjectID.generate()
}
Second one record child case class (very simple) with its writer/reader (both possible way to do it, Macros vs custom in comment)
case class TestRecord() extends Record {}
object TestRecord {
// implicit object TestRecordWriter extends BSONDocumentWriter[TestRecord] {
// def write(testRecord: TestRecord): BSONDocument = BSONDocument()
// }
//
// implicit object TestRecordReader extends BSONDocumentReader[TestRecord] {
// def read(doc: BSONDocument): TestRecord = TestRecord()
// }
implicit def reader = Macros.reader[TestRecord]
implicit def writer = Macros.writer[TestRecord]
}
And then the service itself
class MongoPersistenceService[R <: Record] #Inject()()
(implicit ec: ExecutionContext, tag: ClassTag[R]) {
val collectionName: String = tag.runtimeClass.getSimpleName
def db: Future[DefaultDB] = MongoConnectionWrapper.getMongoConnection("mongodb://127.0.0.1", "27017")
.flatMap(_.database("testDb"))
def collection: Future[BSONCollection] = db.map(_.collection(collectionName))
def persist(record: R): Future[Unit] = {
collection.flatMap(_.insert(record)).map(_ => {})
}
def read(id: BSONObjectID): Future[R] = {
for {
coll <- collection
record <- coll.find(BSONDocument("_id" -> id)).one[R].mapTo[R]
} yield record
}
}
And here's my failing test:
import scala.concurrent.ExecutionContext.Implicits.global
class MongoPersistenceServiceSpec extends WordSpec with Matchers with BeforeAndAfter {
val persistenceService = new MongoPersistenceService[TestRecord]()
"persist" when {
"called" should {
"succeeds when passing a not persisted record" in {
val testRecord = TestRecord()
persistenceService.persist(testRecord)
val persistedRecord = Await.result(persistenceService.read(testRecord._id), Duration(1000, "millis"))
assert(persistedRecord.eq(testRecord))
}
}
}
}
Compiler complains with following messages:
Error:(33, 32) could not find implicit value for parameter writer: reactivemongo.bson.BSONDocumentWriter[R]
collection.flatMap(_.insert(record)).map(_ => {})
Error:(33, 32) not enough arguments for method insert: (implicit writer: reactivemongo.bson.BSONDocumentWriter[R], implicit ec: scala.concurrent.ExecutionContext)scala.concurrent.Future[reactivemongo.api.commands.WriteResult].
Unspecified value parameters writer, ec.
collection.flatMap(_.insert(record)).map(_ => {})
Error:(39, 57) could not find implicit value for parameter reader: reactivemongo.bson.BSONDocumentReader[R]
record <- coll.find(BSONDocument("_id" -> id)).one[R].mapTo[R]
Error:(39, 57) not enough arguments for method one: (implicit reader: reactivemongo.bson.BSONDocumentReader[R], implicit ec: scala.concurrent.ExecutionContext)scala.concurrent.Future[Option[R]].
Unspecified value parameters reader, ec.
record <- coll.find(BSONDocument("_id" -> id)).one[R].mapTo[R]
Anybody has a idea what I might be missing?
I'm still new to scala so help me find the issue in here.
I already tried writing custom BSONWriter/BSONReader instead of the BSONDocumentWriter/BSONDocumentReader provided here without success.
In short, Scala doesn't search everywhere throughout your code for implicits. They need to be brought into scope, either through imports, passing them as parameters, or, as I would suggest here, through a context bound:
class MongoPersistenceService[R <: Record : BSONDocumentReader : BSONDocumentWriter]
This is a kind of short hand for (a) requiring that an implicit BSONDocumentReader[R] (and writer) can be found at the time that the class is constructed with a specific R, and (b) bringing those implicits into scope within the class's implementation so they can be implicitly passed to other methods like insert.
In order to fulfill that new requirement (a) you may have to import TestRecord._ within your test.
Your persist function doesn't have access to the fact that you have defined those implicit functions. The signature should be something like this:
def persist(record: R)(implicit def writer: BSONDocumentWriter[R]): Future[Unit]
And wherever you call persist and pass a TestRecord make sure that the implicit functions are in scope.
Related
Consider a trait that performs "encoding" of arbitrary objects:
trait Encoder[R] {
def encode(r: R): Array[Byte]
}
Assuming encoding of primitive types is known and custom types can be encoded by defining up a "serializer":
trait Serializer[T] {
def serialize(r: T): Array[Byte]
}
we can implement a macro for case class encoding by simply looping the fields and looking up type serializers implicitly. Here's a dummy implementation that looks up the serializer for R itself (in reality we look up serializers for case class field types):
object Encoder {
implicit def apply[R <: Product]: Encoder[R] = macro applyImpl[R]
def applyImpl[R: c.WeakTypeTag](c: blackbox.Context): c.Expr[Encoder[R]] = {
import c.universe._
c.Expr[Encoder[R]](q"""
new ${weakTypeOf[Encoder[R]]} {
override def encode(r: ${weakTypeOf[R]}): Array[Byte] =
implicitly[_root_.Serializer[${weakTypeOf[R]}]].serialize(r)
}
""")
}
}
Now define a base "processor":
abstract class BaseProcessor[R: Encoder] {
def process(r: R): Unit = {
println(implicitly[Encoder[R]].encode(r).length)
}
}
And try to use it:
case class Record(i: Int)
object Serializers {
implicit def recordSerializer: Serializer[Record] =
(r: Record) => Array.emptyByteArray
}
import Serializers._
class Processor extends BaseProcessor[Record]
This fails to compile with:
// [error] Loader.scala:10:22: could not find implicit value for parameter e: Serializer[Record]
// [error] class Processor extends BaseProcessor[Record]
// [error] ^
// [error] one error found
However the following do compile:
class Processor extends BaseProcessor[Record]()(Encoder[Record]) // Compiles!
object x { class Processor extends BaseProcessor[Record] } // Compiles!
I can't really understand why this happens, looks like it has something to do with the Processor definition being a top level definition, since as soon as I move it inside a class/object everything works as expected. Here's one more example that fails to compile:
object x {
import Serializers._ // moving the import here also makes it NOT compile
class Processor extends BaseProcessor[Record]
}
Why does this happen and is there any way to make it work?
To make it work you can try one of the following:
1) add context bound
implicit def apply[R <: Product : Serializer]: Encoder[R] = macro applyImpl[R]
def applyImpl[R: c.WeakTypeTag](c: blackbox.Context)(serializer: c.Expr[Serializer[R]]): c.Expr[Encoder[R]] = {
import c.universe._
c.Expr[Encoder[R]](q"""
new Encoder[${weakTypeOf[R]}] {
override def encode(r: ${weakTypeOf[R]}): Array[Byte] =
$serializer.serialize(r)
}
""")
}
2) make macro whitebox
3) put implicit instance recordSerializer: Serializer[Record] to the companion object of Record rather than some object Serializers to be imported
4) maybe your example is easier than actual use case but now it seems you don't need macros
implicit def apply[R <: Product : Serializer]: Encoder[R] = new Encoder[R] {
override def encode(r: R): Array[Byte] = implicitly[Serializer[R]].serialize(r)
}
my classes look like this
trait Value[T] {
def get:T
}
I have implementations of this, for example
class StringValue(value : String) extends Value[String] {
override def get : String = value
}
class NumberValue(value : Int) extends Value[Int] {
override def get: Int = value
}
The problem is that I need to make jsonFormat for this types in order to save it to MongoDB.
I stucked for two days but still cannot figure out how to make it work
As for the provided .nullable which returns Reads for Option (generic type), you need to first enforce that the type parameter of Value is itself provided the required instances of Reads and Writes.
So for a generic Reads[Value[T]] the minimal def would be as bellow.
def valueReads[T](implicit underlying: Reads[T]): Reads[Value[T]] = ???
Similarily, for Writes[Value[T]] (or OWrites if it needs to be restricted to JSON object, and thus BSON document), the minimal definition would be as following.
def valueWrites[T](implicit underlying: Writes[T]): Writes[Value[T]] = ???
// or
def valueOWrites[T](implicit underlying: Writes[T]): OWrites[Value[T]] = ??? // don't define both as implicit to avoid conflict
Then the implementation depends on the way you want to represent Value[T] as JSON.
Considering the following JSON representation:
{ "_value": ... }
... then the Reads would be something as bellow.
implicit def valueReads[T](implicit underlying: Reads[T]): Reads[Value[T]] =
Reads[Value[T]] { json =>
(json \ "_value").validate(underlying).map { t: T =>
Value(t)
}
}
Similarily, the OWrites[Value[T]] would be as following.
implicit def valueWrites[T](implicit underlying: Writes[T]): OWrites[Value[T]] =
OWrites[Value[T]] { value => Json.obj("_value" -> value) }
Obviously, these implicits need to be in the implicit scope, either by being defined in the Value companion object, or by being explicitly imported if defined elsewhere.
Using context bounds in scala you can do stuff like
trait HasBuild[T] {
def build(buildable: T): Something
}
object Builders {
implict object IntBuilder extends HasBuild[Int] {
override def build(i: Int) = ??? // Construct a Something however appropriate
}
}
import Builders._
def foo[T: HasBuild](input: T): Something = implicitly[HasBuild[T]].build(1)
val somethingFormInt = foo(1)
Or simply
val somethingFromInt = implicitly[HasBuild[Int]].build(1)
How could I express the type of a Seq of any elements that have an appropriate implicit HasBuild object in scope? Is this possible without too much magic and external libraries?
Seq[WhatTypeGoesHere] - I should be able to find the appropriate HasBuild for each element
This obviously doesn't compile:
val buildables: Seq[_: HasBuild] = ???
Basically I'd like to be able to handle unrelated types in a common way (e.g.: build), without the user wrapping them in some kind of adapter manually - and enforce by the compiler, that the types actually can be handled. Not sure if the purpose is clear.
Something you can do:
case class HasHasBuild[A](value: A)(implicit val ev: HasBuild[A])
object HasHasBuild {
implicit def removeEvidence[A](x: HasHasBuild[A]): A = x.value
implicit def addEvidence[A: HasBuild](x: A): HasHasBuild[A] = HasHasBuild(x)
}
and now (assuming you add a HasBuild[String] for demonstration):
val buildables: Seq[HasHasBuild[_]] = Seq(1, "a")
compiles, but
val buildables1: Seq[HasHasBuild[_]] = Seq(1, "a", 1.0)
doesn't. You can use methods with implicit HasBuild parameters when you have only a HasHasBuild:
def foo1[A](x: HasHasBuild[A]) = {
import x.ev // now you have an implicit HasBuild[A] in scope
foo(x.value)
}
val somethings: Seq[Something] = buildables.map(foo1(_))
First things first, contrary to some of the comments, you are relying on context bounds. Requesting an implicit type class instance for a T is what you call a "context bound".
What you want is achievable, but not trivial and certainly not without other libraries.
import shapeless.ops.hlist.ToList
import shapeless._
import shapeless.poly_
object builder extends Poly1 {
implicit def caseGeneric[T : HasBuilder] = {
at[T](obj => implicitly[HasBuilder[T]].build(obj))
}
}
class Builder[L <: HList](mappings: L) {
def build[HL <: HList]()(
implicit fn: Mapper.Aux[builder.type, L, HL],
lister: ToList[Something]
) = lister(mappings map fn)
def and[T : HasBuilder](el: T) = new Builder[T :: L](el :: mappings)
}
object Builder {
def apply[T : HasBuilder](el: T) = new Builder(el :: HNil)
}
Now you might be able to do stuff like:
Builder(5).and("string").build()
This will call out the build methods from all the individual implcit type class instances and give you a list of the results, where every result has type Something. It relies on the fact that all the build methods have a lower upper bound of Something, e.g as per your example:
trait HasBuild[T] {
def build(buildable: T): Something
}
I am currently struggling with spray-json writing a protocol for my data model. For deserialization of JSON data to my data transfer objects, a DAO has to be contacted to check if an appropriate object exists, otherwise a DeserializationException should be thrown.
So far, I have the following:
object MyJsonProtocol extends DefaultJsonProtocol {
implicit object MyDtoJsonFormat extends RootJsonFormat[MyDto] {
override def write(obj: MyDto): JsValue = // Serialization implementation
override def read(json: JsValue): MyDto = {
// parse the JSON, get some parameters, let them be a, b, c
dtoLookup(a, b, c) match {
case Some(dto: MyDto) => dto
case None => throw new DeserializationException("Cannot retrieve object from DAO")
}
}
}
def dtoLookup(a: SomeType, b: SomeOtherType, c: YetAnotherType)(implicit dao: MyDAO): Option[MyDto] = {
// lookup the MyDTO with the dao instance
}
}
My test looks like the following:
class MyJsonProtocolTest extends FlatSpec with Matchers {
implicit val MyDAO = // some test instance, can be a mock object
"The protocol" should "serialize a DTO" in {
val dto: MyDTO = ...
dto.toJson.compactPrint should be("{...}")
}
}
However, the compiler complains that it cannot find the implicit MyDAO when trying to compile the MyJSONProtocol. In When testing Spray services with Scalatest, how to introduce implicit values? I asked yesterday, I was suggested to pass in the implicit parameter directly into the method, but I cannot do this here because the read method is defined in the RootJsonFormat.
When I call the dtoLookup method directly from my test code, it succeeds.
So, how do I get the MyDAO instance into my special JSON format?
One option is to make the implicit parameter a constructor parameter to one of the classes being used. This might require that you turn one of your objects into a class. Then you can make an get method on the companion object of that class that uses an implicit in scope to construct the class with the desired argument.
This doesn't really have to do with spray or scalatest, rather it's just an issue with implicits and implicit scope. Here's a simplified version:
object MyJsonProtocol {
implicit object MyDtoJsonFormat {
def read(x: Int) = dtoLookup
}
def dtoLookup(implicit x: Int) = x + 1
}
And you might consider changing that to:
class MyJsonProtocol(implicit x: Int) {
implicit object MyDtoJsonFormat {
def read(x: Int) = dtoLookup
}
def dtoLookup = x + 1
}
object MyJsonProtol {
def get(implicit x: Int) = new MyJsonProtocol
}
And then you can use this with an implicit in scope:
class MyJsonProtocolTest {
implicit val x = 5
val proto = MyJsonProtol.get
val myReadValue = proto.MyDtoJsonFormat.read //6
}
You can read about the rules for implicit scopes here, especially relevant might be the "Where do Implicits Come From" section.
Just when I thought I understood the basics of Scala's type system... :/
I'm trying to implement a class that reads the contents of a file and outputs a set of records. A record might be a single line, but it could also be a block of bytes, or anything. So what I'm after is a structure that allows the type of Reader to imply the type of the Record, which in turn will imply the correct Parser to use.
This structure works as long as MainApp.records(f) only returns one type of Reader. As soon as it can return more, I get this error:
could not find implicit value for parameter parser
I think the problem lies with the typed trait definitions at the top, but I cannot figure out how to fix the issue...
// Core traits
trait Record[T]
trait Reader[T] extends Iterable[Record[T]]
trait Parser[T] {
def parse(r: Record[T]): Option[Int]
}
// Concrete implementations
class LineRecord[T] extends Record[T]
class FileReader[T](f:File) extends Reader[T] {
val lines = Source.fromFile(f).getLines()
def iterator: Iterator[LineRecord[T]] =
new Iterator[LineRecord[T]] {
def next() = new LineRecord[T]
def hasNext = lines.hasNext
}
}
trait TypeA
object TypeA {
implicit object TypeAParser extends Parser[TypeA] {
def parse(r: Record[TypeA]): Option[Int] = ???
}
}
trait TypeB
object TypeB {
implicit object TypeBParser extends Parser[TypeB] {
def parse(r: Record[TypeB]): Option[Int] = ???
}
}
// The "app"
object MainApp {
def process(f: File) =
records(f) foreach { r => parse(r) }
def records(f: File) = {
if(true)
new FileReader[TypeA](f)
else
new FileReader[TypeB](f)
}
def parse[T](r: Record[T])(implicit parser: Parser[T]): Option[Int] =
parser.parse(r)
}
First off you must import the implicit object in order to use them:
import TypeA._
import TypeB._
That's not enough though. It seems like you're trying to apply implicits dynamically. That's not possible; they have to be found compile time.
If you import the objects as above and change the records so that the compiler finds the correct generic it will run fine:
def records(f: File) = new FileReader[TypeA](f)
But then it may not be what you were looking for ;)
The problem is that the return type of your records method is basically FileReader[_] (since it can return either FileReader[TypeA] or FileReader[TypeB]), and you don't provide an implicit argument of type Parser[Any]. If you remove the if-expression the return type is inferred to FileReader[TypeA], which works fine. I'm not sure what you're trying to do, but obviously the compiler can't select implicit argument based upon a type that is only known at runtime.
1) Using type with implicit inside as type parameter - doesn't bind this implicit to the host type, to do this change objects to the traits and mix them instead of generalizing (type-parametrizing):
def records(f: File) = {
if(true)
new FileReader(f) with TypeA
else
new FileReader(f) with TypeB
}
2) The parser should be in scope of function that calls parse. So you may try smthg like that:
def process(f: File) = {
val reader = records(f);
import reader._
reader foreach { r => parse(r) }
}
PlanB) Simpler alternative is to define type-parameter specific implicit methods inside the AppMain (or some trait mixed in), but it will work only if TypeA/TypeB is known on compile time, so records method can return concrete type:
implicit class TypeAParser(r: Record[TypeA]) {
def parse: Option[Int] = ???
}
implicit class TypeBParser(r: Record[TypeB]) {
def parse: Option[Int] = ???
}
def process[T <: TypeAorB](f: File) =
records[T](f).foreach(_.parse)
def recordsA[T <: TypeAorB](f: File) = new FileReader[T](f)
Here is, I think, the full set of modifications you need to do to get where I think you want to go.
import scala.io.Source
import java.io.File
import reflect.runtime.universe._
// Core traits
trait Record[+T]
trait Reader[+T] extends Iterable[Record[T]]
trait Parser[-T] {
def parse(r: Record[T]): Option[Int]
}
// Concrete implementations [unmodified]
class LineRecord[T] extends Record[T]
class FileReader[T](f:File) extends Reader[T] {
val lines = Source.fromFile(f).getLines()
def iterator: Iterator[LineRecord[T]] =
new Iterator[LineRecord[T]] {
def next() = new LineRecord[T]
def hasNext = lines.hasNext
}
}
sealed trait Alternatives
case class TypeA() extends Alternatives
object TypeA {
implicit object TypeAParser extends Parser[TypeA] {
def parse(r: Record[TypeA]): Option[Int] = ???
}
}
case class TypeB() extends Alternatives
object TypeB {
implicit object TypeBParser extends Parser[TypeB] {
def parse(r: Record[TypeB]): Option[Int] = ???
}
}
class ParseAlternator(parserA: Parser[TypeA], parserB: Parser[TypeB]) extends Parser[Alternatives] {
def parse(r: Record[Alternatives]): Option[Int] = r match {
case x: Record[TypeA #unchecked] if typeOf[Alternatives] =:= typeOf[TypeA] => parserA.parse(x)
case x: Record[TypeB #unchecked] if typeOf[Alternatives] =:= typeOf[TypeB] => parserB.parse(x)
}
}
object ParseAlternator {
implicit def parseAlternator(implicit parserA: Parser[TypeA], parserB: Parser[TypeB]): Parser[Alternatives] = new ParseAlternator(parserA, parserB)
}
// The "app"
object MainApp {
import ParseAlternator._
def process(f: File) =
records(f) foreach { r => parse(r) }
def records(f: File): Reader[Alternatives] = {
if(true)
new FileReader[TypeA](f)
else
new FileReader[TypeB](f)
}
def parse[T](r: Record[T])(implicit parser: Parser[T]): Option[Int] =
parser.parse(r)
}
The gist of it is: all of this would be completely classsical if only your parse instance did not have to pattern-match on a generic type but dealt directly with an Alternative instead.
It's this limitation (inherited from the JVM) that scala can't properly pattern-match on an object of a parametric type that requires the reflection & typeOf usage. Without it, you would just have type alternatives for your content (TypeA, TypeB), which you would add to a sealed trait, and which you would dispatch on, in an implicit that produces a Parser for their supertype.
Of course this isn't the only solution, it's just what I think is the meeting point of what's closest to what you're trying to do, with what's most idiomatic.