Generic Spray-Client - scala

I'm trying to create a generic HTTP client in Scala using spray. Here is the class definition:
object HttpClient extends HttpClient
class HttpClient {
implicit val system = ActorSystem("api-spray-client")
import system.dispatcher
val log = Logging(system, getClass)
def httpSaveGeneric[T1:Marshaller,T2:Unmarshaller](uri: String, model: T1, username: String, password: String): Future[T2] = {
val pipeline: HttpRequest => Future[T2] = logRequest(log) ~> sendReceive ~> logResponse(log) ~> unmarshal[T2]
pipeline(Post(uri, model))
}
val genericResult = httpSaveGeneric[Space,Either[Failure,Success]](
"http://", Space("123", IdName("456", "parent"), "my name", "short_name", Updated("", 0)), "user", "password")
}
An object utils.AllJsonFormats has the following declaration. It contains all the model formats. The same class is used on the "other end" i.e. I also wrote the API and used the same formatters there with spray-can and spray-json.
object AllJsonFormats
extends DefaultJsonProtocol with SprayJsonSupport with MetaMarshallers with MetaToResponseMarshallers with NullOptions {
Of course that object has definitions for the serialization of the models.api.Space, models.api.Failure and models.api.Success.
The Space type seems fine, i.e. when I tell the generic method that it will be receiving and returning a Space, no errors. But once I bring an Either into the method call, I get the following compiler error:
could not find implicit value for evidence parameter of type
spray.httpx.unmarshalling.Unmarshaller[Either[models.api.Failure,models.api.Success]].
My expectation was that the either implicit in spray.json.DefaultJsonProtocol, i.e. in spray.json.StandardFormts, would have me covered.
The following is my HttpClient class trying it's best to be generic:
Update: Clearer/Repeatable Code Sample
object TestHttpFormats
extends DefaultJsonProtocol {
// space formats
implicit val idNameFormat = jsonFormat2(IdName)
implicit val updatedByFormat = jsonFormat2(Updated)
implicit val spaceFormat = jsonFormat17(Space)
// either formats
implicit val successFormat = jsonFormat1(Success)
implicit val failureFormat = jsonFormat2(Failure)
}
object TestHttpClient
extends SprayJsonSupport {
import TestHttpFormats._
import DefaultJsonProtocol.{eitherFormat => _, _ }
val genericResult = HttpClient.httpSaveGeneric[Space,Either[Failure,Success]](
"https://api.com/space", Space("123", IdName("456", "parent"), "my name", "short_name", Updated("", 0)), "user", "password")
}
With the above, the problem still occurs where the unmarshaller is unresolved. Help would be greatly appreciated..
Thanks.

Spray defines a default marshaller for Either[A,B] if a Marshaller[A] and Marshaller[B] are in defined scope inside the MetaMarshallers trait. But, going the other direction requires an Unmarshaller. You will need to define an in-scope Unmarshaller for Either[Failure, Success]. This cannot be coded without specific knowledge of the expected response and what the strategy will be for choosing whether to unmarshall a response as a Left or as a Right. For example, let's say you want to return a Failure on a non-200 response and a Success from a 200 json response body:
type ResultT = Either[Failure,Success]
implicit val resultUnmarshaller: FromResponseUnmarshaller[ResultT] =
new FromResponseUnmarshaller[ResultT] {
def apply(response: HttpResponse): Deserialized[ResultT] = response.status match {
case StatusCodes.Success(200) =>
Unmarshaller.unmarshal[Success](response.entity).right.map(Right(_))
case _ => Right(Left(Failure(...)))
}
}
Update
Looking deeper into this, the problem appears to be that the default eitherFormat in spray.json.StandardFormats is not a RootJsonFormat, which is required by the default JSON unmarshaller defined in spray.httpx.SprayJsonSupport. Defining the following implicit method should solve the issue:
implicit def rootEitherFormat[A : RootJsonFormat, B : RootJsonFormat] = new RootJsonFormat[Either[A, B]] {
val format = DefaultJsonProtocol.eitherFormat[A, B]
def write(either: Either[A, B]) = format.write(either)
def read(value: JsValue) = format.read(value)
}
I have an working example gist that hopefully explains how you would use this. https://gist.github.com/mikemckibben/fad4328de85a79a06bf3

Related

How to match a Case Class containing a Parameter with Generic Type

I have an interesting Problem matching a Case Class in Scala....
I am using Akka and I have functionality that I will use in every Actor in my System, so created a Base Class for my Actor and I try to Match that Command there....
My Command looks like the following...
sealed trait ReportCommand extends ProcessCommand
final case class onReport(key: Key, replyTo: ActorRef[ResponseBase[State]]) extend ReportCommand
while I constructed Base Class so that it might be used from different Actors, onReport is delivered to Base Actor as generic parameter to be used in pattern match with a case class ...
abstract class BaseActor[E: ClassTag, R <: ReportBase[STATE], COMMAND](signal: TypeCase[R]) {
private val report = signal
def base[B <: E: ClassTag](cmd: E, state: STATE)(f: B => ReplyEffect[COMMAND, STATE]): ReplyEffect[COMMAND, STATE] =
cmd match {
case report(report) =>
Effect.reply(report.replytTo)(new ResponseBase[STATE]{
override def state: STATE = state
})
}
}
First if you think this construct will not work, it works, I have another Command (which I didn't place here) which does not have a generic parameter in the Command Class and above snippet is able to match that Snippet.
Now when I first try this code, Shapeless is complained about there is no mapping to ActorRef for Typeable of TypeCase, so after researching the internet I found I have to do the following....
implicit def mapActorRef[T: ClassTag]: Typeable[ActorRef[T]] =
new Typeable[ActorRef[T]] {
private val typT = Typeable[T]
override def cast(t: Any) : Option[ActorRef[T]] = {
if(t==null) None
else if(t.isInstanceOf[ActorRef[_]]) {
val o= t.asInstanceOf[ActorRef[_]]
for {
_ <- typT.cast(myClassOf)
} yield o.asInstanceOf[ActorRef[T]]
} else None
}
}
def myClassOf[T: ClassTag] = implicitly[ClassTag[T]].runtimeClass
implicit def responseBaseIsTypeable[S: Typeable] : Typeable[ResponseBase[S]] =
new Typeable[ResponseBase[S]] {
private val typS = Typeable[S]
override def cast(t: Any) : Option[ResponseState[S]] = {
if(t==null) None
else if(t.isIntanceOf[ResponseBase[_]]) {
val o = t.asInstanceOf[ResponseBase[_]]
for {
_ <- typS.cast(o.state)
} yield o.asInstanceOf[ResponseBase[S]]
} else None
}
}
Now after this changes I don't receive any Exceptions from Shapeless but case report(report) is not matching, I have no idea how we get a reasoning from Scala why it decide it does not match. During my debugging session I observed the following.
I am using the Akka's Ask Pattern to communicate with this actor...
val future : Future[BaseActor.ResponseBase[Actor.State]] = actorRef.ask[BaseActor.ResponseBase[Actor.State]](ref =>
Actor.onReport(key, ref)
)
now if I observe the cmd object that BaseActor receives, I see that 'ask' Pattern of the Akka change ActorRef in the onReport Command class to an ActorRefAdapter, of course ActorRefAdapter is a subclass of an ActorRef but I am not sure what I defined in the implicit for mapping ActorRef to TypeCase can deal with that stuff but I can't figure a way to change implicit to be aware of the Subtypes....
Unfortunately ActorRefAdapter is private to package package akka.actor.typed.internal.adapter so I can't define an extra mapping for ActorRefAdapter.
So can anybody see why Scala is not matching over my Shapeless <-> TypeCase configuration and give me some tips...
Thx for answers...
Your instance Typeable[ActorRef[T]] is incorrect.
Why did you decide to substitute a ClassTag in typT.cast(myClassOf)? This can't be meaningful.
I guess you used something like "No default Typeable for parametrized type" using Shapeless 2.1.0-RC2
If your gole is to make case report(replyTo) matching then you can define
implicit def mapActorRef[T: Typeable]: Typeable[ActorRef[T]] =
new Typeable[ActorRef[T]] {
private val typT = Typeable[T]
override def cast(t: Any): Option[ActorRef[T]] = {
if (t == null) None
else util.Try(t.asInstanceOf[ActorRef[T]]).toOption
}
override def describe: String = s"ActorRef[${typT.describe}]"
}
The problem is that this instance is also bad. Now case report(replyTo) is matching too much.
val actorTestKit = ActorTestKit()
val replyToRef = actorTestKit.spawn(ReplyToActor(), "replyTo")
import BaseActor._ // importing implicits
import shapeless.syntax.typeable._
val future: Future[BaseActor.ResponseBase[Actor.State]] = replyToRef.cast[ActorRef[Int]].get.ask[BaseActor.ResponseBase[Actor.State]](ref =>
1
)(5.seconds, system.scheduler)
Await.result(future, 10.seconds) // ClassCastException
A legal instance of the type class Typeable can be defined not for every type.
Providing instances for (concrete instantiations of) polymorphic types (where well defined) is pretty much the whole point of Typeable, both here and in Haskell.
The key phrase in the above is "where well defined". It's well defined in the case of non-empty container-like things. It's clearly not well defined for function values.
https://github.com/milessabin/shapeless/issues/69
ResponseBase is a non-empty container-like thing. But ActorRef is like a function T => Unit, so there shouldn't be a Typeable for it
trait ActorRef[-T] extends ... {
def tell(msg: T): Unit
...
}
You should reconsider your approach.

Magnet pattern for Scala MongoDB driver

The documentation describes using the magnet pattern to get implicit conversion to BSON types. See on this page http://mongodb.github.io/mongo-java-driver/4.1/driver-scala/bson/scala-documents/. I have tried defining an implicit object that extends BsonTransformer, but it fails to find a codec for the type. Am I missing something / has someone got this working? Sample code below, assume the insert method is being called.
case class CustomType(specialString: String)
implicit object TransformCustomType extends BsonTransformer[CustomType] {
def apply(value: CustomType): BsonString =
BsonString(value.specialString)
}
lazy val db: MongoDatabase = client.getDatabase(dbName).withCodecRegistry(DEFAULT_CODEC_REGISTRY)
lazy val testCollection: MongoCollection[CustomType] = db.getCollection[CustomType](collectionName)
def insert: Future[Completed] = testCollection.insertOne(CustomType("a")).toFuture
Error -
org.bson.codecs.configuration.CodecConfigurationException: Can't find a codec for class com.bla.BlaClass$CustomType.
*note that I am aware this can be done with
val codecRegistry = fromRegistries(fromProviders(classOf[CustomType]))
but I am just using this example to ask to learn the magnet pattern for a messier case.
If there is an implicit BsonTransformer[T] (instance of type class BsonTransformer) in a scope then there are implicit conversions
T => CanBeBsonValue (CanBeBsonValue is a wrapper over BsonValue),
(String, T) => CanBeBsonElement (CanBeBsonElement is a wrapper over BsonElement, which is a "tuple" of String and BsonValue),
Iterable[(String, T)] => CanBeBsonElements (CanBeBsonElements is a wrapper over Iterable[BsonElement])
defined in BsonMagnets (CanBeBsonValue, CanBeBsonElement, CanBeBsonElements are magnets 1 2).
Then Document can be created via factory methods
def apply(elems: CanBeBsonElement*): Document
def apply(elem: CanBeBsonElements): Document
So try
val doc = Document("a" -> CustomType("bb"))
val testCollection: MongoCollection[Document] = ???
testCollection.insertOne(doc)

Why cannot pass function name as argument in this case?

This is a question regarding Akka Cookbook chapter 10's recipe on "Enveloping actor". Can someone please explain why we cannot just pass "headers" function as argument but instead have to pass it as "headers _" in creating the envelopingActor?
The original code can be found here: https://github.com/PacktPublishing/Akka-Cookbook/tree/master/Chapter10/src/main/scala/com/packt/chapter10
package com.packt.chapter10
import java.util.UUID
import akka.actor.{Actor, ActorRef, ActorLogging, ActorSystem, Props}
import Envelope._
object Envelope {
type Headers = Map[String, Any]
case class Envelope[T](msg: T, headers: Headers = Map.empty)
}
class EnvelopingActor(nextActor: ActorRef, addHeaders: Any => Headers) extends Actor {
def this(nextActor: ActorRef) =
this(nextActor, _ => Map())
override def receive: Receive = {
case msg => nextActor ! new Envelope(msg, addHeaders(msg))
}
}
class EnvelopeReceiver extends Actor with ActorLogging {
override def receive: Receive = {
case x => log.info(s"Received [$x]")
}
}
object EnvelopingActorApp extends App {
val actorSystem = ActorSystem()
val envelopeReceiver = actorSystem.actorOf(Props[EnvelopeReceiver], "receiver")
val envelopingActor = actorSystem.actorOf(
Props(classOf[EnvelopingActor], envelopeReceiver, headers _))
envelopingActor ! "Hello!"
def headers(msg: Any) = Map(
"t" -> System.currentTimeMillis(),
"cId" -> UUID.randomUUID().toString
)
}
If you try to remove the _ you may see an error like this:
Error:(379, 59) missing argument list for method headers in object EnvelopingActorApp
Unapplied methods are only converted to functions when a function type is expected.
You can make this conversion explicit by writing headers _ or headers(_) instead of headers.
The thing is that Props is a generic container for any set of actor parameters. Particularly it doesn't know beforehand how many parameters there will be. So it uses following signature:
def apply(clazz: Class[_], args: Any*): Props
You may see that args has type of Any*. This * means that number of arguments is variable and each argument is of the most generic type available in Scala - Any, which is not a function type.
As to why Scala compiler does not automatically transforms methods to function types in such case - I think the answer it is type safety. Imagine it did and then at some point you had headers that took no parameters (so headers was effectively a call of the method) and later you modified it to take one parameter. Since Any matches any type, compiler can't notice that this is actually a breaking change (you've changed an in-place-call to passing just a function that might be called later). If the parameter is strongly typed with some function type, the compiler can notice the difference and thus can safely do transformation automatically.

could not find implicit value for evidence parameter of type ^

I am trying to write a test for a Post request
here is my code :
val request = CreateLinkRequest(token = Some(validToken),billing_ref_id = Some("123"), store_id = Some("123"), agent_id = Some("123"))
val endPoint = Uri(this.serverRootUrl + "path").toString
val post = Post(endPoint, request)
val pipeline = jsonAsStringPipeline(post)
val responseContents = Await.ready(pipeline, atMost = callMaxWaitDuration)
But this doesnt compile, I keep getting this error :
Error:(156, 20) could not find implicit value for evidence parameter of type spray.httpx.marshalling.Marshaller[CreateLinkSpec.this.CreateLinkRequest]
val post = Post(endPoint, request)
^
Error:(156, 20) not enough arguments for method apply: (implicit evidence$1:
spray.httpx.marshalling.Marshaller[CreateLinkSpec.this.CreateLinkRequest])spray.http.HttpRequest in class RequestBuilder.
Unspecified value parameter evidence$1.
val post = Post(endPoint, request)
^
What does this mean?
How can I fix it ?
EDIT:
this is the json in the body :
{ token:"123", billing_ref_id:"123", store_id:"123", agent_id:"123"}
and this is the object for it in the code:
private final case class CreateLinkRequest(
token: Option[String] = Some(validToken),
billing_ref_id: Option[String] = Some(Random.alphanumeric.take(10).mkString),
agent_id: Option[String] = Some(Random.alphanumeric.take(3).mkString),
store_id: Option[String] = Some(Random.alphanumeric.take(3).mkString)
)
You are trying to call Post method which takes implicit Marshaller as parameter. Note that implicit parameters don't have to be provided as long as the compiler can find one in the scope (check this for more info about implicits: Where does Scala look for implicits?)
However, your code doesn't have any implicit Marshaller defined so the compiler doesn't know how to convert your case class into the HttpEntity.
In your case you want it to be converted into the HttpEntity with Content-Type: application/json. To do that you just need to define: implicit val CreateLinkRequestMarshaller: Marshaller[CreateLinkRequest]. This tells scala how to convert your case class into the HttpEntity.
You also want it to be passed to the context as JSON, so we are going to define our JsonProtocol, i.e. MyJsonProtocol.
package test
import spray.http.HttpEntity
import spray.http._
import spray.json._
import spray.httpx.marshalling.{Marshaller, MarshallingContext}
import test.TestMarshaller.CreateLinkRequest
object MyJsonProtocol extends DefaultJsonProtocol {
implicit def createLinkRequestFormat: JsonFormat[CreateLinkRequest] = jsonFormat4(CreateLinkRequest)
}
object TestMarshaller extends App {
import MyJsonProtocol._
case class CreateLinkRequest(token: Option[String], billing_ref_id: Option[String], store_id: Option[String], agent_id: Option[String])
implicit val CreateLinkRequestMarshaller: Marshaller[CreateLinkRequest] = new Marshaller[CreateLinkRequest] {
override def apply(value: CreateLinkRequest, ctx: MarshallingContext): Unit = {
val entity = HttpEntity(MediaTypes.`application/json`, value.toJson.compactPrint)
ctx.marshalTo(entity)
}
}
// Here goes your test
}
Note that you can define these implicits somewhere else, e.g. a package and then just import it in the test class. This would be better because you will certainly want to reuse the Marshaller.

Passing a Scala type to a function

I'm trying to implement basically the same thing that is discussed here, but in my specific situation, it's not working.
Currently I have a function that validates a JSON response from our server. The problem is, it has the JSON type hardcoded into the method:
def fakeRequest[A: Writes](target: () => Call, requestObject: A): Any = {
route(FakeRequest(target()).withJsonBody(Json.toJson(requestObject))) match {
// ... stuff happens
Json.parse(contentAsString(response)).validate[GPInviteResponse]
^
Note the hardcoded GPInviteResponse type.
So, to make this a totally generic and reusable method, it would be great to pass in the type that is being validated.
I tried this:
def fakeRequest[A: Writes, B](target: () => Call, requestObject: A, responseType: B): Any = {
route(FakeRequest(target()).withJsonBody(Json.toJson(requestObject))) match {
// ... stuff happens
Json.parse(contentAsString(response)).validate[B]
^
That almost works, but I get a No Json deserializer found for type B. Makes sense, so I changed it to:
def fakeRequest[A: Writes, B: Reads](target: () => Call, requestObject: A, responseType: B): Any = {
But, now I get No Json deserializer found for type controllers.GPInviteResponse.type. So the question is: Is it possible to pass a type like this (or is there some other magic to make it work)?
There is a deserializer defined for the type... I've re-read it half a dozen times to make sure there's no typo:
case class GPInviteResponse(inviteSent: Boolean, URL: Option[String], error: Option[GPRequestError] = None) {
def this(error: GPRequestError) = this(false, None, Option(error))
}
object GPInviteResponse {
implicit val readsInviteResponse = Json.reads[GPInviteResponse]
implicit val writesInviteResponse = Json.writes[GPInviteResponse]
}
Edit
Included below is a simplified test case that demonstrates the problem. At the moment, this won't compile (error is shown below). I think I understand why it won't work (vaguely) but I have no idea regarding a solution. My theory as to why it won't work: While the provided type GPInviteRequest does have an implicit Reads/Writes method, Scala cannot make the connection that an instance B is actually a GPInviteRequest so it concludes that B does not have Reads/Writes.
case class GPInviteResponse(inviteSent: Boolean)
object GPInviteResponse {
implicit val readsInviteResponse = Json.reads[GPInviteResponse]
implicit val writesInviteResponse = Json.writes[GPInviteResponse]
}
class TestInviteServices extends PlaySpecification {
"try to validate a type" in {
tryToValidate(GPInviteRequest(true))
}
def tryToValidate[B: Reads, Writes](i: B) = {
val json = Json.toJson(i).toString
Json.parse(json).validate[B].isSuccess must beTrue
}
}
The above test yields:
[error]
/Users/zbeckman/Projects/Glimpulse/Server-2/project/glimpulse-server/test/application/TestInviteServices.scala:46:
No Json serializer found for type B. Try to implement an implicit
Writes or Format for this type. [error] val json =
Json.toJson(i).toString [error] ^ [error]
/Users/zbeckman/Projects/Glimpulse/Server-2/project/glimpulse-server/test/application/TestInviteServices.scala:133:
No Json deserializer found for type controllers.GPInviteResponse.type.
Try to implement an implicit Reads or Format for this type. [error]
fakeRequest(controllers.routes.GPInviteService.invite, i,
GPInviteResponse) match { [error] ^
Your error message:
No Json serializer found for type B. Try to implement an implicit Writes or Format for this type.
In this function, how is the toJson method supposed to know how to serialize your type B?
def tryToValidate[B: Reads, Writes](i: B) = {
val json = Json.toJson(i).toString
Json.parse(json).validate[B].isSuccess must beTrue
}
You haven't pulled in a writer/reader into scope, the compiler doesn't know where to look for one and that's why it's telling you to implement one. Here's a quick solution
case class GPInviteResponse(inviteSent: Boolean)
object GPInviteResponse {
implicit val format = Json.format[GPInviteResponse]
}
def tryToValidate[B](i: B)(implicit format: Format[B]) = {
val json = Json.toJson(i).toString
Json.parse(json).validate[B]
}
Note: using the format method is equivalent to defining both a reads and writes.
Now, there is an implicit formatter for B in scope, so the compiler knows where to find it and injects it into the validate method which takes a reader implicitly:
// From play.api.libs.json
def validate[T](implicit rds: Reads[T]): JsResult[T] = rds.reads(this)
Edit
You can add type parameters to your function and then reference them in the validate[T] method, like so:
// Define another case class to use in the example
case class Foo(bar: String)
object Foo {
implicit val format = Json.format[Foo]
}
def tryToValidate[B, C](implicit f1: Format[B], f2: Format[C]) = {
val j1 = """{"inviteSent":true}"""
val j2 = """{"bar":"foobar"}""" //
Json.parse(j1).validate[B]
Json.parse(j2).validate[C]
}
// Example call
tryToValidate[GPInviteResponse, Foo]
Try it this way :
def tryToValidate[B](i: B)(implicit format : Format[B]) = {
val json = Json.toJson(i).toString
Json.parse(json).validate[B].isSuccess must beTrue
}